Deprecated: mysql_connect(): The mysql extension is deprecated and will be removed in the future: use mysqli or PDO instead in /storage/content/49/145849/famitracker.com/public_html/forum/classes/dbHandler.php on line 29 FamiTracker
Login:
Menu:
Post: Author:
FamiTracker > General > Bug Reports & Feature Requests > Cleanup option to merge duplicate pattern data Owner: rainwarrior New post
Page 1 of 1 Sort:  
Cleanup option to merge duplicate pattern data Posted: 2012-11-30 18:18  (Last Edited: 2012-11-30 18:21) Reply | Quote
rainwarrior

Avatar

Member for: 4150 days
Location: Canada
Status: Offline

#42461
When I was trying to conserve the size of an exported file for a project that had copy-paste frame duplication I put the following code in OnEditRemoveUnusedPatterns. It detects any exact duplicate patterns and replaces them with a reference to the lowest numbered copy of that pattern. After doing this, the rest of the code to remove unused patterns takes effect, eliminating the duplicates from the data. This can cut down the size of an FTM, but can also reduce the size of the exported data as well (where it's more important).

I dunno if it belongs in OnEditRemoveUnusedPatterns specifically, since it may be desired to remove unused patterns without eliminating duplicates as well, so maybe it should be two separate options.

[url=http://pastebin.com/LtntqfHz]http://pastebin.com/LtntqfHz

Posted: 2012-12-02 01:45 Reply | Quote
jsr
Administrator

Avatar

Member for: 5925 days
Location: Sweden
Status: Offline

#42487
This kind of optimization is actually a part of the NSF exporter since 0.4.0, there it also has the advantage of finding and eliminating duplicated patterns across channels and songs.

But I can see the use of this code when using other kinds of export methods. I'll add it as a separate option as I also think that it's a bit different from just cleaning up unused patterns.

Thanks!

_______________________
Programmer and developer
Posted: 2012-12-02 05:20 Reply | Quote
rainwarrior

Avatar

Member for: 4150 days
Location: Canada
Status: Offline

#42490
Ah, I didn't realize the NSF export does this now. Interesting. I had written this bit back when I was using 0.3.6 I think.

Also, the pattern comparison can probably be done with a single memcmp on the whole patterns, I'm not sure why I wrote it row-by-row (maybe the raw pattern isn't accessible from the interface?).

Posted: 2012-12-02 22:33 Reply | Quote
Patashu

Avatar

Member for: 3749 days
Location: Sydney, Australia
Status: Offline

#42505
If you use memcmp, you assume that two patterns can't be the same logically unless they are physically. (Don't know if this is true or not for Famitracker, but it's the kind of thing that could change by bug or deliberation)

Posted: 2012-12-03 00:40 Reply | Quote
rainwarrior

Avatar

Member for: 4150 days
Location: Canada
Status: Offline

#42513
Yeah, there are cases where a field can hide differences and look/act the same. However, for the common copy-paste case I was trying to target, they should normally be the same.

Though... I am literally doing memcmp on each row, so it's effectively no different than a memcmp on the pattern.

I guess it would be safest to do a smart compare row-by-row. It would probably run in a reasonable amount of time anyway.

Page 1 of 1 Sort: