I was wondering about this, since Novedicus posted this text in one of his posts about the Dr. Yundong hack. Well, I thought that every game had the same engine. So, what's the differences about them? and why it was necessary to make them, and why they didn't stick with the S1 engine for the rest of the games? Also, is this also applicable with sound drivers? This one, I surely know why, but I don't know how.
Because, they needed to evolve the engine over time and optimize it. The internals of the Sonic 1 game engine is honestly complete garbage. (Tho, Sonic 3K's isn't that good either :VVVVV)
You have to understand the Mega Drive was only a year or two old before Sonic 1 was first written, the programmers were not 100% familiar with the machine, and mostly had to rely on their knowledge from previous games they've programmed (many if not all of which were on more primitive hardware), there are external hardware factors involved too, such as cartridge circuit and memory design, and time restraints would play a role. Spoiler Let's take the level engine as an example, Sonic 1's original design were 256x256 chunks holding 16x16 blocks holding 8x8 tiles. This creates a nice balance between memory usage and CPU time, when moved onto Sonic 2 the chunk sizes were scaled down to 1/4 the size at 128x128, this would allow for more flexible layout placement, additionally the chunks can be reused whilst keeping the familiarity of the chunk structure down, this leads to saving memory for the chunks. The downfall to this is the layout size being now a x4 the size larger. Other technical feats include the idea of utilising the spare bits in the chunk's mappings for a second path of collision's solidity status, where before loops were achieved by chunk swapping, and were carefully restricted to only a few safe instances probably due to discomfort with how successful it would be (I recall an interview with Naka where he mentions the initial programming of loops and how Sonic would fly through the loop if he went too fast, so indeed there were teething problems). The collision detection and level drawing mechanisms employed look up tables with pre-calculated X and Y positions for accessing the level data, this improves CPU time at the expense of ROM space as the CPU requires less instructions to achieve the same result. In addition, mechanisms for displaying multiple pieces as individual sprites linked directly to a single object would help to save on SST RAM and CPU time. The art tiles and 16x16 blocks were compressed using an LZSS compression mechanism known as Kosinski, originally the tiles were compressed via an RLE/Huffman/Entropy format which was specifically designed for tile pixels in a nybble fashion, and the blocks were compressed via an RLE/increment/decrement VDP map specific format known as Enigma, these formats were preferred as these were supplied by SEGA as part of the library, and these formats were specially designed for art/map tiles. They moved over to Kosinski's LZSS format, because LZ algorithms have the advantage of being fast to decompress, and usually result in smaller data due to the data having a high chance of repeated streams. Music was moved over to the Z80 sub-CPU to save on main 68k CPU time, and the music was compressed in an LZ based algorithm with automatic 0 markings known as Saxman, the purpose of compression was to save on ROM space, as the music would need to be decompressed into Z80 RAM anyway. With cartridge technology improving, DMA transfers from ROM directly to VRAM were less likely to fail, so direct DMA transfers were adopted, where before with Sonic 1 all data had to be transferred from RAM (which is why Sonic's art is loaded to RAM before it's DMA'd off). The above are all advantages which make the game easier and quicker; to design, to program, to lag less, and to save on RAM and ROM space which can be utilised for more important aspects. As we move onto Sonic 3&K, the layout mechanism was improved, every row would now be indexed via word pointers to save on CPU time via reduced instruction requirements, and can save on RAM space with shorter rows, allowing for larger height levels in expense, likewise, it allows the swapping out of individual layout rows to be accomplished quicker. New techniques are developed and employed, specifically on the dynamic art front, where specific objects will load individual frames of art into the same VRAM space, often to save on overall VRAM, much of which would come at the cost of ROM space as the art would need to be uncompressed for a fast and direct transfer, however, this is 1993/94, ROM cartridge space is larger and cheaper, more widely available for abuse. Many objects had long-word pointers directly to the routine they would run, thus saving on CPU instructions to multiply an ID to a long-word, and the priority link ID would be swapped for a link word address for direct access to the RAM priority list. The draw code for the level data was changed, the mechanism involves loading level tiles into RAM during display period, and DMA flushing them off during V-blank where time is most critical. This can slow the overall engine down, but as just mentioned, it saves time during V-blank and allows H-blank to interrupt and access the VDP without interrupting anything. The previous version takes up too much V-blank time, so much so you can see the delay in question in Sonic 1 and 2, whenever you're moving too fast in both axis, you'll see the top right block will display the previous frame as the VDP has already drawn the tiles before the 68k got around to writing the new ones into VRAM. The reduced V-blank time opens up room for more DMA transfers and raster effects, both of which contribute to the game's design. The sound driver was further changed, kept in Z80 RAM, though the music, was moved out of Z80 RAM and instead kept in ROM and referenced via the window directly, this would increase the ROM space, however as mentioned before, ROM space is not really an issue at this stage, likewise the Saxman compression didn't really have much of an impact, but if you think about it, the SMPS (or SMFD as we've recently discovered) technically is a compression system, it has LZ based looping capabilities, as well as reductions to repeated notes/timers, at any rate, this leaves more Z80 RAM space for more Z80 code for specific SFX routines, and more space for pointer entries, as well as a slew of other minor changes to the driver. Later games such as the Crackers/Chaotix series of games employed linked list systems for chaining up objects into categories, or "families", thus allowing you to search a specific chain for specific instances without having to search the entire object RAM and checking if they are of a specific type (Sonic 2 had employed such a mechanism before, by moving the rings into their own specific RAM, that could "technically" count as it achieves the same CPU time saving strategy to a degree). Crackers also employed the idea of object's ID's within a layout being automatically multiplied by x4 to save the CPU having to perform the multiplication/shift. The level mechanism is almost identical with exception to the last 10 collision array blocks which appear to flip in an alternating fashion, I'm not too entirely sure what they aimed to gain from that though, the art, blocks and chunks for the levels' FG and BG are split into their own individual parts. FG's parts are loaded, followed by BG's parts, these are dynamically loaded such that one will load right after another, and the mappings are carefully adjusted dynamically, this would allow for easy swapping out of art to/from VRAM without having to carefully plan it out. The V-blank/H-blank routines are set dynamically based on the game mode itself by directly jumping to the routine, rather than wasting CPU time referencing a table of pointers, and jumping to the correct one based on the game mode current. Crackers also includes a new simpler LZSS compression routine, it has similarities to the Saxman format, though it has no automatic 0 markings, and it's retrace/copy (offset/length) ranges can be adjusted to obtain the best size, this being of simpler design compared to Kosinski allows for faster decompression at the expense of compression size. Although, it was only used for the SEGA logo and the map screens. A lot of the Crackers engine was a rewrite technically though, but it was built upon the ideas and techniques learnt over the Sonic games (as well as other titles), so in a way it does count. There are a few numerous other things which I could point out, but they aren't coming to mind right now. But as you can see, they've learnt from their mistakes, made improvements, gained experience and time, not only that but you can see more programmers joined the team as time progressed, bringing their own skills to the table. It can all best be summed up as REFINERY. You'll be saying the same about Crackers/Chaotix later when you end up working on a better engine, then you'll do the same to that one too~ Remember, the engines were "good for their time", Sonic 1 was revolutionary in 91 you know.
Novedicus isn't lying when he says the internals are garbage... =P But seriously, considering the context of when they were made with the time constraints they had and the limitations in tools and equipment and understanding, they didn't do that bad. What we look through now is the work of several programmers from the late 80's/early 90's, reverse-engineered and adapted for toolchains that run on systems a few orders of magnitude more powerful than was available at that time, and with a lot of time and exploration and effort put in for analysis. As much as we can call a lot of the code crap now, we do have to keep that in mind for perspective. Now, as to your question, the engines of the various Sonic games of the era both are and aren't the same engine. What do I mean, you may be asking yourself. Well, to simplify matters, the engines of S3K, S2, SCD, and Chaotix can all trace their roots back to the same core code that made Sonic 1. In many ways, the games were all built as evolutions of the Sonic 1 engine. By that same token of explanation, though, that also makes the engines quite different. Code can be ported from engine to engine, but require adaptations (sometimes extensively) to actually function on those engines. They are each distinct in characteristics and behaviors, sharing similar roots and similarities in structuring and operation, but have widely differing characteristics on handling things. Components can be interchanged, but often require tooling to adapt to the frameworks of the target engine. In the case of the Yundong project, it's very likely that the programmers involved (which include Novedicus) decided that the behaviors and components of S3K with components of Chaotix/Crackers better-fit their needs than S1 did. That's not to say S1 wouldn't have been able to do the job; just that S3K provided a platform that was naturally more able to fill the roles needed without quite so much work. As always, mileage may vary depending on the needs of a project, so while they decided S3K was suitable for their project, that shouldn't suddenly mean that the S1 engine is bad. Just not what best-fit their needs. Sound drivers are interchangeable, so long as the interfacing code is adjusted to communicate with the driver properly. Like other components, if adaptations are made, they can be used. Mind, this is a quick and simple top-level explanation. There's *far* more depth that could be gone into, but I'll leave others to go there if they'd like. In the meantime, I'll leave you with a fun tidbit that helps illustrate a bit about how what we know now changes things: A few years back, when I had free time still, a friend and I decided to see what we could do in one week to just up the Sonic 1 engine's performance. Within that week, with a lot of booze, we had: * added optional 32x support, able to be enabled by a build-time switch * replaced Nemesis and Kosinski with a highly-optimized and tooled variation of Lempel-Ziv compression algorithm tailored for performance and excellent compression for the hardware * replaced several calculation-heavy areas of engine code with look-up tables when performance was quantitatively better * DMA optimizations * Sound driver replacement for a Z80-based custom driver (unreleased) Those are just some of the things we did within the span of a week. After that, the engine was running so fast that parts we hadn't improved needed timing adjustments, because certain expectations of loading times were grossly off. We were having title card corruption from level art loading too fast. Ultimately, we just stopped working on the project, and the source is probably floating around on some drive in their or my possession, but it shows that with what we know now, which is leagues better in understanding and analysis than probably what the original programmers even had while writing the code, we can make the old code look like trash. That makes perspective all the more important, though. Respect the programmers for what they did at the time, even if now we understand and have analyzed their code to depths far beyond the measures they took.
I always tend to forget to remember the context of why the engine was the way it was when talking about it, so fair enough, y'all. >=P As for why I chose the engine I did for Dr. Yundong, I mainly was looking to just move on to a more optimal base from Sonic 1 after being frustrated with how poorly aged it is these days. The Sonic Crackers part comes in with the level rendering engine, which uses multiple block buffers instead of a singular one, so it doesn't have to check if it's a row or column and all that jazz. Otherwise, it's just Sonic 3K's engine, stripped down to its core and optimized just a bit more (it uses a linked list object handler to run objects, made by Natsumi, which lessens the CPU load some, for one, also the priority SST is just a direct pointer to the priority slot or objects). I've mentioned before that I changed the chunk size to 32x32 so that more precise level design can be achieved easily, and also so that I could use the same rendering engine for the overworld engine. With the sound driver, I went with AMPS+Dual PCM, because Dr. Yundong uses PCM SFX in some areas, and also wanted dat sweet Jester Stream Technology(tm) to keep certain SFX from sounding like complete crap on hardware/good emulators. At the end of the day, choose whatever fits for you best.
I choose nothing besides asking how the "chunk flips" works. ;u; I think I'm being off-topic in my own tread. Anyway, thanks for clarifying everything, guys!
With the chunks being 32x32, it's a lot easier to flip chunks and takes less CPU time. Basically, when getting a block from a chunk that's flipped, I have it swap the offset in the chunk data so that it reads the other block in the row/column of the chunk, and then also flip the block itself. Applies to both rendering and collision detection. The chunk IDs are 16 bits, too. For level editing, I use a modded (a half assed mod at that) SonLVL that supports 16 bit chunk IDs and chunk flipping. The disadvantage to this is that level layout data greatly expands, so you'd have plan put how you use your ROM space if you'd like to do something like this.
