Community disassemblies

We all know how poor the situation with our current disassemblies; one set, Hivebrain 2005, Xenowhirl 2007, and Puto's old Sonic 3 & Knuckles disassemblies, are really outdated by todays standards. But, the other set; Git line of disassemblies either are not user friendly, use AS, are incomplete and not updated regularly, suffer from other major issues, or combination of the above! So really, we do not have one disassembly which caters both to newbies, advanced programmers, or someone who wants to port things between games. In fact every disassembly fails on all aspects, some less, some more! We clearly need better!

We hope to bring SSRG community disassemblies, maintained by the community as a solution; We will be using VASM to assemble our project, document it very well, but most importantly, use naming schemes as similar to other games. This would consist for Sonic 1, Sonic 2, Sonic 3, and Sonic & Knuckles disassemblies, as well as a solution allowing you to combine the Sonic 3 or Sonic 2 with Sonic & Knuckles disassemblies to make S3K and S2K respectively. We hope to get some members from the community to do active development of these disassemblies with us, as there is a lot of work to do! The disassemblies will be done completely from scratch and strive for their own style over supporting existing disassemblies. LazloPsylus will help to organize this project, and I will lead it and ensure its done right, as well as contributing it to myself as much as I have time for.

But alas, time doesn't fall from the sky, so working on this alone will not just be feasible. So we need you. Yes, you! You can poke me if you want to help along. You'll be required to have good grasp on the 68k, as well as basic understanding on the Sonic engine to help along.

 

I would love to help out on this project. As I have shown in the past, I have a pretty good grasp on 68000 Assembly, the MegaDrive, and the Sonic engine (even if I haven't put any of my knowledge to good use), so I should be capable of helping out.

I have a question, though. Are the disassemblies planned to build bit perfect ROMs?

 

I doubt we will keep the ROM bit-perfect. The reason is because some code in the games splits an object in half, and therefore there is no easy way to keep the file structure simple, yet all the code easy to find. Just look at Sonic 1 Git and you see what I mean. I think we would reorganize the code so that the engine functions are easy to find and dont interfere with object code.

 

To be honest, I would prefer if the code were to be reorganized, so that's cool. Thanks.

 

Certainly I could not help too much because I am quite busy with my own project at very limited ocassions, so I try to always focus and use my time well. But if you allow me to throw some extra tips here...

- Objects and Sprites are not the same thing. I am not born in English, but I am very convinced, just like a lot of other people here, that "Objects" are what the CPU executes as a whole "being", and "Sprites" are the pieces sent to VDP. I think "Objects" and "Sprites" are very different. So, could you stick to the proper terminology, and DO NOT name the "Objects" as "Sprites"?

- According to the Sonic 2 Alpha's ROM, the original source codes had all their ASM split into many different files, so they could be built separately and linked to the final ROM image using a special tool - Just like any normal source code do nowadays. And, while the git disasm of Sonic 1 gets a lot of blame, it is my personal favourite just because of its organization, because regardless of being stupid or incomplete at times, with equates and macros that sometimes look like nothing but "mental notes", it allows to do certain stuff quite fast for those who are used to work with Sonic 1 or anything else at a technical level. When getting started with mine project, I knew I would not use most of the original objects, so, after deleting their entries in the pointers table, I went to _incObj and got rid of any unnecessary files in no more than 5 minutes, and finally I fixed all broken INCLUDEs according to the Sonic.Log. If I attempted to do the same in any Sonic 2 disassembly, I would have to open s2.asm, and scroll from start to the end, slowly trying to hunt any unnecessary code as I see it; and while some people would have no problems on doing so, in all honesty, HOW MUCH people you think is going to do such an effort?. That's why keeping everything in a same file, at least to me, seem like a big turn down at the time of reorganizing or cleaning my ROM.
If the ROM you guys generate with your custom disassembly is not bit-perfect, is there any reason for keeping the Sonic 1's ROM structure messy as that? I don't think so. That's why, I think, the split structure would not be bad for this time. And, perhaps, to make it even better, it might be good if the _incObj equivalent had each object within a folder, with all the parts there (tiles, maps, etc.); faster solution, taking the map editors into consideration (because not needing to navigate through folders to pick the necessary files, turns the process slightly less tedious).

- I can be wrong, but I think a great lot of people doesn't feel "encouraged enough" to do better coding. And none of the disassemblies actually help to the matter. The main pro of hacking Sonic 1, against the other games, is the fact that the engine is very clean, has no "surprise" exceptions, and because its simplicity it is much easier to extend some features to the project's needs without collateral effects. The trouble here is, that it seems like many people stick to the ROM's internal structure and/or the engine capabilities and doesn't want to go -much- further (i.e. "I can not do this because the engine does not support this X function", or, crate a new zone, and still call it SYZ, GHZ, etc...); now, take a look at Sonic 1 Git's id_GHZ, id_LZ... why do it have to be like this? how will a major mass of hackers change their mind and redo new zones from scratch, if they have the original names all over the place? Why not id_Zone00, id_Zone01?. Besides, if the ROM is not going to be bit-perfect, why not to get rid of some exceptions and checks, and extend the original arrays to get the same result? For example: Instead of writing an exception to display "Final" as the title card for SBZ3, why not to move the "final" zone to a new ID, and let the title card object to read the text from this new ID?
Less exceptions, less fixed IDs, and less default/pre-defined shit MAY mean a lot of improvement, but, again, I can be wrong.

In conclusion, I'm just saying that it should put more emphasis in USAGE, rather than in explaining how the original ROMs work. I don't think anyone besides the Git maintainers do care about bit-perfection, which is rather pointless for the hacker as it is willing to do a lot of changes later. Keep it like a real source code. The hacker could have a better and more specialized workspace for its goal. And thus achieve much better results.

 

I don't recall people referring objects as sprites here. The only time I saw objects being referred to as sprites were in Esrael's disassemblies. Other than that, people usually called them objects.

 

I really like this idea! I might consider adding tips in as it goes along, but as it stands for now,I have other things to attend to. I'm glad to see something like this finally escaping the ideas phase :3

P.S. This new disassembly setup should probably label every method and ram space with the older disassemblies naming in mind, to help with using older tutorials for the new assembly, something like the Github Sonic 2 assembly.

 

If you don't care about bit-perfection, why not make an engine clone with a language that people actually know, for a platform that people actually use?

 

We will scrap the old and honestly outdated ideas in favor of better naming scheme, cleaner code, and better compatibility between different games. If we respected the rules of the older disassemblies, we'd just be reinventing over making the wheel round. We will create proper documentation for these disassemblies which would help people adapt to them from whatever disassembly they are used to.

We fully intend to create a naming scheme which relies on terms that are accurate and widely known. Especially the Sonic 2 and 3 disassemblies tend to replace "object" or "entity" with "sprite" when not necessary, and this bothers me as well. Sprites indeed are only really related to VDP displaying objects, and sprite pieces are the smaller parts which build up the sprite itself.

There is no special tool needed; practically any assembler can do this already. Even SNASM68 for all of its flaws has decent support for including files.

Sonic 1 gets flack mostly because the files are split poorly, without enough thought behind them, and some objects are split into two because of a subroutine in middle of them! It does make an initiative for how split files could work, but it does a poor job at it on its own. However, even bigger issue is, that there is no pointers to what data is in which file. You could have 5 subroutines in separate file, and then have to search for one of them among tens of files just to find the right one. It tends to waste a lot of time if you do not know where to find what, which is really bad. But the plus side is, if you know where everything is, its much more efficient to work on something where you need to edit a lot of places in the ROM.

I have done that exact thing; deleting every single object from S3K disassembly (and a lot of other code, too). To be honest it was much easier than you make it out to be, but yes it is still a lot of work, and this is something we like to avoid.

We will not make the data split before we are sure we have made all the big code editing things. These include creating variable names, changing global lable names, etc. because its simply easier to do this when the data is in a single file. However, we will eventually be creating a split structure. I am not yet sure for what we go, but I have an idea of having all the data a certain object uses in a single directory, all the data one level uses in a single directory, etc.

This is indeed wrong; Sonic 1's engine is the least clean from the later games. Aside from having a lot of hacks in them, the engine is very well thought out and refined to the finest. Unlike Sonic 1, which contains a lot of unused code, some rather odd design choices, a lot of buggier and slower code, and weirdly programmed objects. Not to mention entirety of SBZ (including FZ). Of course, none of the games are perfect, but Sonic 1 was not really programmed that well (of course, they didn't know if the game was going to take off, so they probably put a lot less effort into it.). However, one of the advantages of Sonic 1 is that its much easier to modify for your own will, because the code is much simpler, and its smaller as a whole.

I dont think a name of a zone internally matters that much for what they can do with it. But I do agree, the disassemblies do not take capabilities of the assembler in mind with creating better experience for the hacker. One thing I think we will eventually do, is make zones a lot easier to add. Rather than changing over 10 arrays, you could just change one array, and add necessary files. This would be done by creating an entry describing few things about the zone which would be necessary, then the assembler would generate proper pointers and include necessary files you had created in order to add a zone.

Its quite a hard one, because you need to know where to draw the line with what you change; with just bit perfect ROM its easy, you add few macros, few variables, change lable names, and call it a day. But if you get to modify the ROM, where is the line? You could go forever about it and end up with an open source hack for all you know. This, of course, is not our goal.

Yes, but it has to be carefully thought what we change. Creating a disassembly from old game like this with specialized engine is not easy, but creating an environment for everyone to use is in a whole new level.

Because we are creating disassemblies?
If I wanted to make an engine on Java or C# or something like that, I would not use the Sonic engine as its base, because its too specialized and buggy. It's been built in the 90's, and it works best for the 90's, not today

 

And yet you're reinventing parts of it, anyway. It seems pointless to do one drastic thing in the name of making things easier, but not any others.

 

Would you elaborate what exactly you mean?

 

Disassemblies are borderline impenetrable, especially when you have something as messy as the Sonic games. Is it any surprise things like Sonic Before/After the Sequel are based on fan engines? You say you want this new line of disasms to 'cater both to newbies, advanced programmers, or someone who wants to port things between games', to justify, not just better labelling, but the lack of bit-perfection, and rearranging object code for consistency. You're practically making another BOOM or ReadySonic, only, instead of features, it's a usability overhaul. Or, is what you're suggesting not an overhaul, but merely some tweaks? It seems odd to not be loyal to the original code, but still remain a disasm. Good labels and untangling some objects can only do so much, so why not go further?

 

I never said anything about not being loyal to original code, but rather, re-organizing it to be better suited for editing and making your own hack out of it. It is going to be a full overhaul in comparison to other disassemblies we have right now. We may tweak small bits of code here and there for sake of usability, but usability alone, the main focus is still to provide sold disassemblies of the games.

 

If I had my two cents in it, the immense craze for bit-perfection is the reason why I can't take the modern disassemblies very seriously. You are severely limiting yourself and the way an game works for the soul purpose of keeping the disassembly as close to the original as possible. The times of research, as far as I'm concerned at least, are completely over with Sonic 1 and 2. If—and that is a big if—you are still concerned about that, nothing stops you from going back to the original disassemblies and taking a look at those.

To give some bigger picture, I think what saxman tried to do with the Boom thing was fine from the concept. His problem was that he majestically failed at marketing his project properly. I also think he went too far with a few things (including the "improved" level layout format).

That being said, I'd definitely like to see more hacking-friendly versions for disassemblies. Entirely free from self-imposed bullshit restrictions.

 

Once I have finished working on my DocuDisASM of Sonic 1, I had planned on making a PC port of it, and then releasing it open source. That is why I require a fully/correctly documented disassembly.

To do that, I would need to have a bit for bit perfect reassembly, this is to ensure that when rewriting aspects of the source (supplying documentations/macro/equate alterations) that I haven't made a mistake. A bit for bit perfect design will help reduce on mistakes, by catching the bulk of them.

Believe it or not, there are things in Sonic 1's disassembly that were not made clear, such as Marble Zone's hidden cycle palette data. The research is not over yet, not until it's done properly, and everything is understood. Once we have done that, we can then move onto unlimiting ourselves, the expressed knowledge of the entire source 100% will help us with that.

 

When it comes to S3K barely 10% are actually documented anywhere. Rest of the game is either unknown or someone knows about it but has made no effort to document it. These disassemblies also strive to improve the current documentation, but also to document a lot more of the games. And indeed, Sonic 1 and Sonic 2 are neither well documented despite everyone believing they are.

 

Again, this ties in with the question what it is you are trying to create here. A disassembly optimized for original hack creations or a disassembly that just wants to be as documented as possible? Go ahead, try and create something that does both just fine, but I can promise you that with an increase of accuracy you will have to make some compromise in the usability section. I'm not even saying that to demotivate anyone interested in such a project. It just is a fact that soundness and completeness have an extremely hard time to co-exist.

Thus I got a feeling you should decide what exactly it is you want to build first, or sooner or later you will run into nasty issues.

 

I'm sure whatever disassemblies that come out will be useful in some way. I'd really like to see a Sonic 1 disassembly like the Git one (with variables and constants and is well documented) but with most of the asm code (aside from stuff under _inc) in one file. Other than that, I'd like to see general documentation improvements from other disassemblies and maybe have bug fixes (not features that someone might not want in their hack like a fix for the spike bug or the spindash in Sonic 1) could be preadded. I can't wait to see the disassembies that come out of this!

Maybe someone can create a Sonic CD disassembly with documented code. It would be very useful even if you couldn't build it.

 

I believe there should be a reorganized disasm due to it being serverly needed and like Natsumi said, if you need a bit perfect disasm go to git/hive and if you want to make a sonic hack easily without having to search around and guess a lot of where some stuff might be then use the more orgnised one. It's really up to the user which one they use and I think it's best if we give people more options then just basically the same disasm only different labels :V

 

Wow. This is just...
I haven't seen something like this in years... This is indeed, the best article I've read so far. Though I may give a suggestion! the usage of AS for assembling. The reason of this..? Mainly, compatibility with Z80! ASM68K can't compile Z80 code...
I hope that this project grows big. If I could, I would contribute myself, but at the moment, the university is a bit overwhelming...
Also, For the sake of documented labels and such... My suggerence is to look in each disassembly; there are some labels that are undocumented in a sonic game but documented on the other and vice-versa. (I know this by expereince...)
Anyways..! Keep up the good job!