ClownMDEmu - The Greatest Mega Drive Emulator Ever (Someday)

Discussion in 'Showroom' started by Clownacy, Jun 23, 2022.

  1. Clownacy

    Clownacy Retired Staff lolololo Member

    Joined:
    Aug 15, 2014
    Messages:
    1,065
    You're right, I forgot that some Mega CD rips lack a CUE file. Thanks for the reminder - that's yet another format that I should add support for.
     
    peachy likes this.
  2. Clownacy

    Clownacy Retired Staff lolololo Member

    Joined:
    Aug 15, 2014
    Messages:
    1,065
    v0.7.1
    Try it in your web browser: clownmdemu.clownacy.com
    Download: https://github.com/Clownacy/clownmdemu-frontend/releases/tag/v0.7.1

    Being that it is currently a weekend, I have some spare time to release another update. Mainly, this update adds support for more Mega CD disc rip formats, but there is another improvement that I wanted to highlight as well:

    Fix Window Icon Blurriness on High-DPI Displays
    A user shared this screenshot of the emulator in a GitHub Issue:

    [​IMG]

    This desktop is running at an extremely high resolution, which is made evident by the high-resolution font rendering. However, when looking at the very top-left of the screenshot, something odd can be seen...

    [​IMG]

    Right next to the crisp, high-resolution font is an awfully blurry, low-resolution icon. I found this to be very strange, as the icon which I made for the emulator supported resolutions up to 1024x1024, and yet here the window is clearly using the 16x16 version of the icon.

    I eventually found that setting the icon of the window is the responsibility of the SDL2 library, and that it was using a peculiar method of extracting "default" icons from the running executable: it would obtain the full path to the executable, use the ExtractIconEx WinAPI function to extract specific large and small icons from it, and then assign those to the window. I found this odd because it seemed unnecessary to obtain the path to the current executable when SDL already has a handle to the executable. Additionally, WinAPI provides functions to obtain icons from the current executable by loading them as resources, which, to my understanding, is the standard method which the majority of people use.

    Icons can exist on their own or in a group. It appears that ExtractIconEx returns lone icons rather than icon groups, and that the icons which it returns are not the ideal resolution for high-DPI displays. Windows relies on icon groups in order to support high-DPI displays, selecting the icon with the appropriate resolution from the icon group to be shown. Because ExtractIconEx was not returning an icon group, Windows was unable to do this, resulting in the 16x16 icon being used at all times.

    SDL2 has a 'hints' system, allowing some degree of configuration. With it, I can force the window to use a particular icon or icon group, instead of relying on SDL2 to choose one automatically. Indeed, doing this fixes the issue, causing an icon with a decent resolution to be shown instead:

    [​IMG]

    But this was not good enough for me: why should I, and every other user of SDL2, have to set a hint to make SDL2 use the correct icon resolution? SDL2 should get it correct by default! So, I took it upon myself to fix SDL2. According to this StackOverflow answer, in situations where Windows itself has to assume an executable's icon (such as when viewing it in File Explorer), it uses the first icon group within it. Using WinAPI's EnumResourceNames function, I made SDL2 do just this. As I had hoped, this fixed the issue completely, ensuring that the best icon resolution is always used!

    With my work done, I submitted it as a Pull Request to SDL2's GitHub repository, and it was promptly merged! When SDL2 v2.30.3 or v2.32.0 is released, expect to see this bugfix included as a part of it! This improvement also benefits the upcoming SDL3, which boasts improved high-DPI support as one of its main features!

    Support Many More Mega CD Disc Rip Formats
    v0.7 only supported rips in the 'BIN+CUE' format. This is a basic format where the BIN file contains a dump of the disc's sectors, and the CUE file contains a transcription of the disc's Table of Contents. While this format does well to preserve data accurately, it uses a lot of space due to two things: 13% of the CD-ROM data being useless junk data, and the CD-DA audio being uncompressed PCM.

    To address this, other formats exist, such as 'ISO+OGG+CUE'. In this format, the CD-ROM data is moved to its own ISO file with the junk data removed, and the CD-DA audio is also split to its own files and encoded in the Ogg Vorbis format. This greatly reduces the size of the rip: for example, the Japanese version of Sonic CD is reduced from 556MiB to 126MiB.

    The downside to such a format is complexity: the rip is no longer a straightforward dump of raw, uncompressed sectors, with each one being exactly 2352 bytes; now, CD-ROM data is in sectors that are 2048 bytes in size, while CD-DA audio is encoded and lacks the notion of sectors altogether. The encoding in particular is a massive complexity, as formats such as Ogg Vorbis are extremely complicated and not very feasible to create a new decoder from scratch for, necessitating the use of pre-existing software libraries instead.

    It is with such libraries that I have extended my emulator to support additional formats! In particular, stb_vorbis, dr_flac, dr_mp3, and dr_wav are used to support Ogg Vorbis, FLAC, MP3, and WAV audio! This should encompass the majority of formats commonly used by Mega CD rips. What is special about the libraries which I have chosen is that they minimise the concerns about portability and licensing, due to them being written in ANSI C and released under extremely-permissive licenses.

    Another format that is now supported is the 'CUE-less' rip: in this format, there are only ISO and audio files, and no CUE file. With no CUE file linking them all together, these files are instead linked through their naming: the ISO file is named 'XXXX.iso', and the audio tracks are named 'XXXX 02.ogg', 'XXXX 03.ogg', etc. If the user makes the emulator load the ISO, then the emulator will automatically search for the audio files and load them too.

    The 'CUE-less' rip in particular is the best format for homebrew developers, as they do not have to maintain a big fragile CUE file, nor do they have to go through the rigmarole of inserting junk data into their ISO file, decoding their audio to raw PCM, and combining it all into a gigantic BIN file.

    With these formats supported, archivists, homebrew developers, and typical end-users alike are all catered to!

    There is one format that still needs to be added, however: CHD. This is a format from MAME, which encodes the audio in FLAC, compresses the ISO in one of several compression schemes, and combines it all into a single file. This format has found popularity relatively recently, being added to many emulators under the RetroArch umbrella. It walks the line between targeting end-users and archivists, as it preserves the original data while still saving space through the use of compression. Unfortunately, the library which is used to decode CHD files is woefully undocumented, preventing me from integrating it into my emulator. I hope to eventually figure out how to use this library so that every common rip format is supported.

    Closing
    With this, the biggest shortcoming of the previous update has been addressed. The emulator should now be well-rounded enough to last people until the next update. I wonder what I will work on next?
     
    Crimson Neo, DeltaW and ProjectFM like this.
  3. Clownacy

    Clownacy Retired Staff lolololo Member

    Joined:
    Aug 15, 2014
    Messages:
    1,065
    (Cross-post from my blog).

    v0.8
    Try it in your web browser: clownmdemu.clownacy.com
    Download: https://github.com/Clownacy/clownmdemu-frontend/releases/tag/v0.8.0.1

    v0.8 is an update that was burdened by attempting a major change to the codebase. This change was meant to greatly improve the emulator's frontend, allowing for the elimination of numerous hacks within its codebase while making new features possible. Unfortunately, this did not go to plan, and all of the work on it cannot be released today. This change was the migration of the frontend from version 2 of the SDL library to version 3.

    This would have been perfect for adding native multi-window support, but deficiencies in the pre-release SDL3 meant that the frontend would also be rendered extremely unstable and buggy. Instead, today's release is a refinement of the emulator based on its prior software stack. While it would be great to announce and show off native multi-window, that feature will have to wait until SDL3 matures. With that out of the way, here are the features that avoided disappointment.

    Dark Title Bar on Windows 11
    [​IMG]

    It is about time! Thanks to a tip from Ewan, the frontend now features a little WinAPI hack to set the title bar to the same colour as the Dear ImGui menu bar. This will only work on Windows 11 due to using a recent API.

    [​IMG]
    Now Windows 11 users no longer have to put up with the ebony-and-ivory look that the emulator previously had.

    Linux AppImage Build
    Linux executables tend not to be anywhere near as universal as Windows executables: their heavy reliance on system libraries and lack of standalone installers make it very difficult to share precompiled software in a form that will work without any hassle on a variety of Linux distributions. Because of this, users either have to compile software from source code, or wait for their distro to provide it as a package. Unfortunately, clownmdemu is not provided by any distro, meaning that compiling from source is the only option. However, this is not a simple process, as it requires recursively checking-out the Git repo and then bootstrapping the build system with CMake - things which beginner programmers are apparently unfamiliar with. All of this combined makes it quite difficult for a Linux user to obtain a useable copy of my emulator.

    This is where AppImage comes in - AppImage is a standard for Linux software which bundles all files (executables, libraries, and data) into a single executable, making it very convenient to share, download, and run. Additionally, software is encouraged to be compiled on the oldest still-supported version of Ubuntu LTS, allowing it to run on a huge range of Linux distributions without risk of glibc incompatibilities. With AppImage, I can produce a Linux build that is easy to download and run while also being compatible with the vast majority of current Linux distributions!

    Supporting AppImage was not particularly difficult: it merely requires that the emulator's build system be capable of installing the software in a way that adheres to some freedesktop.org standards - particularly providing an icon and basic metadata. Likewise, compiling with an old version of Ubuntu was not very difficult, with it only requiring that I mark some CMake scripts as being compatible with older versions of CMake, and also needing to switch from C++20's format library to its backward-compatible middleware version.

    With that done, I am now able to create a clownmdemu AppImage! It is provided alongside the Windows executable in the GitHub releases.

    But why stop there? AppImage and its two competing standards - Flatpak and Snap - all provide catalogues for users to conveniently browse and download compatible software. It is one thing to make my software easy to run, but it would be even better to also make it easy to find! So, I have added clownmdemu to appimage.github.io!

    Fix for Sprite Data Not Being Cached on Prior Scan-Lines
    The ROM-hack 'Gametap Pro Gamer 6' had a couple of splash screens which were not rendering correctly in this emulator:

    [​IMG] [​IMG]
    Hey, that's me!

    The cause of this bug was surprisingly difficult to uncover, taking me several days. Eventually, I found that this issue was not the product of a bug, but rather a deliberate design choice: back when I had implemented sprite scan-line caching, I made it so that it would not cache the sprite data of scan-lines which had already been rendered. In hindsight, this makes absolutely no sense. I think this was a misguided optimisation, but, after all these years, I no longer remember why I implemented it in the first place. Regardless, by removing this anti-feature, this bug is now fixed:

    [​IMG] [​IMG]
    I really am the best!

    Approximation of the YM2612's 'Ladder Effect'
    A feature which purists often mention in relation to the Mega Drive is a curious bug that is present in early models of the console: the 'low-volume distortion' effect which is present in the YM2612 sound synthesiser, but not the YM3438 that is used in later models.

    The exact details of how this bug works do not yet appear to be understood, however the general idea of it is that an offset is applied to samples on one half of the sound wave. Exactly how large this offset is is unclear, but I have seen at least one mention of it being approximately 4 decrements of the YM2612's internal 9-bit mixer value.

    This bug causes quiet sounds to be louder than normal. Many songs and sounds were designed with this behaviour in mind, causing the absence of this bug to make them sound incorrect. For instance, here is a song from Earthworm Jim 2:

    Without the low-volume distortion, the grating whining in the background is absent. According to Tommy Tallarico, this whining is an intended part of the song, rendering its absence a flaw.

    Another major example of this is the After Burner II soundtrack, with three songs bearing distinct differences:

    YM2612:

    YM3438:

    • Red Out has a descending note at the start which is inaudible without the low-volume distortion.
    • Super Stripe has an electric piano-sounding instrument playing some backing notes which are much quieter without the low-volume distortion.
    • After Burner has a lead melody whose notes decay much faster without the low-volume distortion.
    These differences were egregious enough that they simply needed to be addressed, and so I have introduced a rough approximation of this bug into clownmdemu. Given its rough nature, and the fact that this bug does not exist (at least to the same extent) in later Mega Drives, an option to disable this is has also been added.

    [​IMG]
    Have it your way!

    Support for 'KDEBUG' Debug Messages

    It is not often that standards manifest within Mega Drive emulation: there is no standard save state format, no standard layout for modern gamepads, and not even an agreed standard for how to convert Mega Drive colours to modern sRGB RGB888. KDEBUG, however, is an exception:

    KDEBUG is an extension to the Mega Drive's API that provides a way for software to log text for the user to view. While useless for players, this is great for developers, in the same way that C's 'printf' function is: it can be used to make the game report things to the developer that are difficult to determine otherwise, such as how many of a pool's objects are allocated, or how many entries are currently populating a linked-list.

    This API originated in Gens KMod, which is a modified version of the historical Mega Drive emulator Gens. KMod introduced a vast number of debugging utilities to Gens, with this API being one of them. Many years later, this API was adopted by another emulator, BlastEm. This API would go on to be used by Vladikcomper's debugging framework (which is widely used by Sonic the Hedgehog ROM-hacks) as well as SGDK (which is widely used by Mega Drive homebrew). In all, KDEBUG has become quite ubiquitous.

    Indeed, it has become so ubiquitous that I have received multiple requests for KDEBUG support be added to my emulator too.

    The API itself is pretty simple: it uses one of the VDP's unused registers as a character stream output, allowing strings to be logged by feeding each character into the register one after the other. After the last character is sent, a null character (literally the byte 0x00) is sent, which the emulator responds to by presenting the complete string to the user. On a real Mega Drive, writing data to this register does nothing whatsoever, making it a harmless no-op. This allows software which uses the KDEBUG API to remain compatible with platforms which do not support it.

    As with other developer notifications, KDEBUG messages are recorded to the emulator's Log window. By enabling the 'Log to Console' option, the messages will be sent to 'stderr', allowing them to be viewed from the terminal just like a native PC program.

    [​IMG]
    An example of Vladikcomper's debugger logging which chunk data is being loaded.

    Accurate Motorola 68000 Instruction Durations

    CPUs work by processing instructions. These instructions can do things like add two numbers together, perform a multiplication, and so on. Different instruction take a different amount of time to execute... and this is what my emulator has been failing to do for its entire existence.

    The Z80 CPU emulator has had proper instruction durations for a long time now, but that was due to the necessity of it: games rely on the Z80's timing to be correct so that audio samples do not play at the wrong speed. However, for the Mega Drive's main CPU, the 68000, this necessity did not exist: many games worked perfectly with every instruction taking 10 cycles, and so my emulator did just that.

    This remained the case until I overhauled how Z80 interrupts worked, which exposed an issue: in Sonic 2, with each 68000 instruction taking 10 cycles, the Z80 interrupt would consistently occur right in the middle of a long Z80 bus request operation, causing the interrupt to be lost, which meant that the music would play at an uneven, slow speed.

    Rather than mitigate the consequences of this hack by adding another hack, I opted to eliminate the hack by finally implementing proper 68000 instruction durations. This was done by creating a test program which ran the 68000 CPU interpreter in isolation and compared the results to a sprawling series of tests whose data was derived from an extremely accurate microcode-level emulator. This way, I could compare my emulator's behaviour to that of the accurate emulator. This is something that I had already done previously back in late 2021, but I only verified instruction behaviour rather than instruction durations. With this new program, I could verify both!

    All instructions now have accurate durations! With this, this music in Sonic 2 no longer plays at a wonky speed, and games should lag (or not lag) more consistently with how they do on a real Mega Drive.

    True Multi-Window
    Ever since debug menus were first added, they looked like this:

    [​IMG]

    The menus mostly worked, but they had one glaring limitation:

    [​IMG]

    They were all fake! It was not possible to see a 'fake' window outside of the 'real' window! Yes, you could maximise the emulator, but that meant obscuring all of the other programs that you were using, like your web browser!

    The reason that the windows were this way was because they were made with Dear ImGui, which is a library for quickly and easily creating menus. This library works across many different operating systems, rendering its windows inside the program's main window by design, so that it works on video game consoles, which lack a proper windowing system like what a PC has. Unfortunately, this illustrates how Dear ImGui is better-suited for embedding in a video game than for developing a sprawling desktop application.

    Despite this, I continued using Dear ImGui because I hoped that its developers would eventually address this shortcoming and allow for the creation of proper windows on platforms that support it. Indeed, such a feature has already been in development for quite some time!

    [​IMG]

    There was just one problem: it did not work on Linux. Not only was the X11 support extremely buggy, but it was fundamentally incompatible with Wayland, meaning that it would never support the protocol without first undergoing major modifications. This was a deal-breaker.

    For years, I hoped that something would change, but nothing ever did. Even now, Dear ImGui's 'multi-viewport' support is completely unusable on modern Linux.

    Finally, I had waited long enough, and decided that, if Dear ImGui was not going to support multiple windows for me, then I was going to do it for myself. This is the result:

    [​IMG]
    It's finally here!

    At last, all menus are now proper windows! This was achieved by bypassing Dear ImGui and manually creating windows using the underlying API, SDL2. Unlike Dear ImGui, SDL2 supports creating windows on all major operating systems, including Wayland-powered Linux.

    Long ago, I had refactored the frontend so that a window was represented by a class, with which it was easy to create multiple windows. By inheriting from this class, I could create a class which represented a window that used Dear ImGui widgets, and then, by inheriting from that, I could create a class that represented a menu window. In this hierarchy, the main window is a 'window with Dear ImGui', while all other windows are 'menu windows'. Each menu can then inherit from the menu-window class and extend it with a unique interface. By structuring the code this way, all menus share a great deal of code, making it very easy to create new ones. Additionally, by centralising so much code into common classes, I made it easy to apply fundamental modifications to all menus at once! This would be very useful for solving one particular problem...

    Unfortunately, not all platforms support using more than a single window. One platform in particular is one that my emulator only recently gained support for - Emscripten. By making my emulator use native windows, I had broken compatibility with platforms that cannot use them! To address this, I made it so that a menu can be either a real window or a fake window. Due to the aforementioned centralisation of the core menu code, this was very easy to do! As a result, the Emscripten version of the emulator will use Dear ImGui windows, while other platforms will use normal windows!

    Allowing the menus to be used outside of the main window has been a feature that many users have requested over the years, so I am very happy to finally see this fulfilled!

    There is just one problem: these menus are jank. On Windows, Linux, and presumably every other platform, these windows are not considered 'subwindows' of the main window. This means that each one gets a slot on the operating system's task bar, and that un-minimising one window will not cause the rest of them to be un-minimised along with it. These sound like small problems, but they really bring the whole experience down.

    [​IMG]
    This is annoying.

    SDL2, despite flaunting its ability to create multiple windows as one of its main improvements over SDL1, has no API to overcome this. There is one function for designating a 'modal' window, but it does not solve this particular problem. Not to mention, that function is only implemented on Linux; it does absolutely nothing on Windows and macOS, and was only introduced because the Linux port of the Unreal Engine's editor needed it. It is things like this which call SDL2's role as a platform-abstraction library into serious question.

    Because of this, Dear ImGui windows remain the default for now, until something can be done to make native windows less clunky. For those feeling adventurous, native windows can be enabled by editing the frontend's configuration file, which can be found at '%APPDATA%/clownacy/clownmdemu-frontend/configuration.ini' on Windows and '~/.local/share/clownacy/clownmdemu-frontend/configuration.ini' on Linux. Just change the 'dear-imgui-windows' setting from 'on' to 'off', and you are good to go!

    SDL2's successor, SDL3, provides a function for making a window the parent of other windows, which would solve all of these problems. One would think that the obvious thing to do is migrate this project to SDL3 so that it can make use of this feature, and that is exactly what I did! Unfortunately, SDL3 is still a work-in-progress, being far too unstable for regular use. While the day will come that the switch to SDL3 can be made, and the improved native windows can finally be released for everyone to use, that day is not coming any time soon.

    Preliminary Migration to SDL3
    Background
    From the very beginning, this emulator has used the SDL2 library for handling the various differences between operating systems, allowing the emulator to work on any platform with minimal effort. There is one problem with it, however: it is quite old.

    SDL2 as we know it began as SDL 1.3 in the late 2000s. Since its inception, it has maintained a rock-stable programming interface (both API and ABI), which allowed it to continue working with all software that had been built upon it, even after many years of releases, features, and heavy refactoring. This interface also had the unfortunate downside of cementing various poor design decisions into SDL2, forcing developers to deal with them for well over a decade.

    At long last, SDL's developers have opted to shed these years' worth of flaws by developing SDL3, which boasts a refreshed API. Gone are quirks like functions returning 0 to indicate success while others would do it to indicate failure, the filtering of textures being specified by clunkily altering global state, and high-DPI support on Windows being enabled completely differently to how it is for other platforms.

    In addition to a cleaned-up interface, SDL3 boasts new features like a 3D rendering API, file dialogs, and a replacement for the conventional 'main' function that makes software immediately compatible with Emscripten.

    None of this is particularly beneficial to my emulator, since it does not need 3D rendering, it already uses its own custom file dialog logic, and it provides special-case code for Emscripten to work around the incompatibility with 'main' functions. However, by discarding this custom code and offloading the work to SDL3, not only would the codebase be simplified and lightened by several hundred lines of code, but improvements made to SDL3's implementation of these features would directly benefit the emulator, whereas it would remain completely unaffected if it continued using its own implementations.

    What ended up pushing me over the edge, however, was the ability for a window to declare itself as the child of another window. By doing this, the child window is not allocated an icon on the task bar, and it is presented to the user whenever its parent is. This may seem minute, but it goes a long way towards making a multi-window program not feel like a mess to use.

    Wanting to make the emulator's multi-window support the best that it can be, I decided to migrate my emulator from SDL2 to SDL3.

    Migration
    The process of migrating to SDL3 was very complex. Fortunately, the developers of SDL provide extensive documentation for how to port software to the new library, along with a few scripts to automate certain tasks such as renaming functions.

    One of these scripts gave me a great deal of trouble, that being the Coccinelle script. Coccinelle is a program meant for applying patches to source code in a syntax-aware manner, but, on Windows and Arch Linux, it merely crashes when attempting to run the script. It lacks a Flatpak and neither MSYS2 nor Arch Linux provide a package for it, requiring that it be compiled from source, which makes it unclear whether the fault is with the program or merely how it was compiled. The only way that I got the script to run was by installing a recent version of Ubuntu in a virtual machine and installing and running its Coccinelle package from there, which is far more effort than it is worth. I do wish SDL's developers had chosen a more reliable tool.

    After applying the Coccinelle script, I was able to run the accompanying Python scripts without any problems. From there, I consulted the migration documentation to address every compiler error that I encountered until the emulator successfully compiled. Unfortunately, a few bugs had sneaked in along the way, due to some API changes not producing compiler errors despite the issues they caused, such as the 'SDL_Event' struct's 'cdevice' ('controller device') member being renamed 'gbutton' ('gamepad device'), and then a new member being added that was also called 'cdevice' (short for 'camera device' instead). This problem could have been avoided by using a more verbose naming scheme.

    With the emulator now running atop SDL3, I could replace its custom file dialog and 'main' function logic with usage of SDL3's new APIs. I also considered doing the same to replace the audio mixer, but I am uncertain about whether SDL3's mixer features dynamic rate control like mine does, so I have not made the switch.

    Result
    As of now, the SDL3 migration is complete. The codebase is benefitted by this in multiple ways: being free of many lines of code that were made redundant by SDL3, having clearer code due to targetting a better API, and being able to make use of features that did not exist in SDL2. With this, the emulator is prepared for the future!

    ...Unfortunately, it is not prepared for the present: SDL3 is still pre-release software, meaning that it is lacking in polish and stability. A glance at SDL's issue tracker will show many flaws in its current implementation. The worst problem occurs when closing a window. On Windows 11, this causes the desktop to behave erratically, as if the compositor or GPU driver had crashed, while on Linux and Windows XP, the emulator itself crashes. Strangely, this only seems to occur with the software and Vulkan renderers; OpenGL and DirectX appear to work just fine.

    Because of all of this, the SDL3 migration was moved to its own dedicated branch. The main version of the emulator will remain with SDL2 until things improve. Those who are interested in trying the SDL3 port can find its source code here.

    Closing
    I think this is the first time in the project's history that I have implemented features which were either completely reverted or locked behind a hidden setting. Both native window support and the SDL3 migration held this update back for months, so it is a tremendous shame to not see all of that effort pay-off. This has me wondering if it would be worth abandoning SDL in favour of another library, such as GLFW. Unfortunately, SDL is so deep-rooted into the codebase that removing it would involve rewriting a great amount of code. While it is possible, it would not be easy. Still, refactoring is one of my favourite programming tasks, so maybe I will take the plunge. Only time will tell.
     
  4. Clownacy

    Clownacy Retired Staff lolololo Member

    Joined:
    Aug 15, 2014
    Messages:
    1,065
    (Read this post on the blog if it does not look correct here.)

    v1.0
    Try it in your web browser: clownmdemu.clownacy.com
    Download: https://github.com/Clownacy/clownmdemu-frontend/releases/tag/v1.0

    The big day is here! After three years of development, my Mega Drive emulator has reached a basic level of feature-completeness! At this point, it is not unusual to load a game that had never been tried in the emulator before, and for it to run flawlessly! The final vital features to be added were Motorola 68000 instruction durations, save data, and accurate window plane emulation. With those now added, the emulator is finally "complete"!

    Save Data
    One of the final major features for this emulator to support is save data. On the Mega Drive, there are two particular variants of save data: SRAM and EEPROM. Each is a different technology that a cartridge uses internally to store data, with each requiring different code to emulate. SRAM is simpler and more commonly used, so it is the first of the two to be emulated. EEPROM is not currently emulated. EEPROM is only used in a handful of games, so there is not anywhere near as much of a need to emulate it. For the games that do use EEPROM, there are likely ROM-hacks and patches which convert them to SRAM.

    In addition to plainly emulating SRAM, the emulator's frontend also saves and loads the save data to and from a file, preventing it from being lost when the emulator is closed. The save data file is named after the ROM file that was used to create it, and can be found with the emulator's configuration files ('%appdata%/clownacy/clownmdemu-frontend' on Windows, and '~/.local/share/clownacy/clownmdemu-frontend' on Linux). Save data is fairly standard, so I have done what I can to make it compatible with other emulators, however I cannot guarantee compatibility with all emulators, given how there are many different types of SRAM save data (odd bytes only, even bytes only, odd and even bytes, 0x400 bytes total, 0x4000 bytes total, etc).

    Approximate H-Counter and H-Blank Flag Emulation
    The H-counter is a feature that I have dreaded implementing for a very long time, as it is a feature that is deeply related to how the Mega Drive's Video Display Processor internally operates. They essentially give the X and Y coordinates of the pixel on the screen that the VDP is current drawing. Given that my emulator does not recreate how the VDP behaves internally, this makes the H-counter (as well as the V-counter) very awkward to emulate.

    In particular, a real VDP operates by drawing horizontal pairs of pixels called 'slots' one after another in a line from left to right, with lines being drawn in sequence from top to bottom. This process is complicated by the need for the VDP to drive a cathode-ray tube display, timing the beam as it moves not only from left to right and top to bottom, but also from the right back to the left and from the bottom-right back to the top-left. Altogether, this makes for a mess of timings which the H-counter and V-counter are forced to reflect. Because my emulator does not need to drive a CRT, these timings are completely absent.

    However, accurate H-counter and V-counter values are not entirely necessary: in theory, a lot of games would be fine with a vague approximation of them. Indeed, several games benefitted from the V-counter approximation that I added several months ago. Given that an inaccurate V-counter has proven to be better than no V-counter at all, I finally opted to implement the H-counter.

    The way that the H-counter is emulated is by measuring the total number of cycles that occur per scan-line and dividing it into 208 parts (the H-counter can count up to 256, but it seems that about 48 values are skipped), with the H-counter reporting which of the 208 'slices' that the current cycle fits in.

    In addition to allowing games to detect roughly how far into a scan-line the VDP currently is, implementing the H-counter also allows me to implement the H-Blank flag, which is a bit that can be read from the VDP's control port to determine whether the VDP is currently 'blanking' horizontally (not displaying anything as it switches from ending one line to beginning another). This is needed in order for the game 'Another World' to boot.

    Battletoads Landing Cable
    For months, this issue has been sitting on the issue tracker. In it, it is described how, in just one game, a certain set of sprites is missing.

    [​IMG]
    The issue report came with this handy illustration.

    Given that drawing arbitrary curvy lines is a complicated process, especially when using sprites, I assumed that the cable was drawn using H/V counter trickery. However, even once basic H/V counter emulation was added, the problem still remained. Stumped, I opened the emulator's sprite debuggers to see what the game was trying to do.

    [​IMG]

    The sprite list debugger immediately presented me with a series of cable sprites! So the game was using conventional sprites to render the cable! But that raised the question of why they were not visible. To help figure this out, I opened the emulator's other sprite debugger; the sprite plane viewer.

    [​IMG]

    At first glance, it may look like the cable sprites are still missing, but they are in fact barely visible at the very bottom of the viewer! It became obvious to me what the game was doing: by pushing the sprites past the bottom of the screen, it was relying on the Y coordinate overflowing to have the sprites come out of the top of the screen!

    To emulate this behaviour is simple: take the Y coordinates of all sprites, and bitwise AND them by 0x1FF. This has the same effect as computing the remainder of a division by 0x200 (512). 512 is the effective range of the X and Y coordinates, which is why the sprite plane viewer is a defined square and not an infinitely large 2D space. Given that I already knew that the coordinates should have been constrained to a range of 0-512, it is embarrassing that I had forgotten to apply this restriction to the emulator itself and not just its debuggers.

    With that mistake corrected, the cable finally shows up as intended!

    [​IMG]
    Aren't debuggers great?

    libretro Revival
    After a year and a half of neglect, the libretro frontend has been brought up-to-date with the stand-alone frontend, being updated to the latest version of the emulation core and made to leverage its new features! New to the libretro frontend are support for 6-button joypads, persistent save data, and Mega CD games, along with all of the accumulated accuracy improvements that have been made to the emulation core!

    From now on, the plan is to keep the libretro frontend up-to-date with the standard frontend, not letting it fall into disrepair like it had previously. The reason for this is that the libetro frontend serves its own distinct purpose: while the standard frontend is specifically designed to assist in the development of homebrew and ROM-hacks, the libretro frontend is for actually playing games! For the standard frontend, features such as net-play, video and audio recording, and shaders are out-of-scope, and will likely never be implemented, while the libretro frontend already features all of these and more when paired with RetroArch!

    In the future, I hope to add ClownMDEmu to the libretro organisation's build-bot, allowing the emulator to be downloaded and installed directly from within RetroArch! One of the main reasons that I had not attempted this before was that the emulator was still missing major features, but, as the v1.0 label indicates, I no longer feel that that is the case.

    Drag-and-Drop Mega CD Games
    Save states and Mega Drive games can be loaded by dragging-and-dropping them onto the emulator window, however, this feature did not work for Mega CD games, as the emulator would try to load them as Mega Drive games instead. This has been corrected: the frontend now attempts to detect if the file is a Mega CD game, and correctly treats it as one if so.

    The code for this feature actually originated in the libretro frontend, as Mega Drive games and Mega CD games have to share the same loading logic in that frontend. Figuring that the code could be used to improve the standard frontend's drag-and-drop support, I moved the logic into a small utility function in the common frontend code repository, allowing it to be shared by both frontends!

    Massively-Optimised CPU Idling
    The Mega Drive, and Mega CD especially, are complicated machines: featuring multiple CPUs which share access to RAM and other hardware. The CPUs cannot access them at the same time, so when one is using them, the others must wait for their turn. The way that this was done in my emulator was less than ideal.

    Code:
    static cc_u16f SyncMCDM68kForRealCallback(const ClownMDEmu* const clownmdemu, void* const user_data)
    {
       const Clown68000_ReadWriteCallbacks* const m68k_read_write_callbacks = (const Clown68000_ReadWriteCallbacks*)user_data;
    
       return CLOWNMDEMU_MCD_M68K_CLOCK_DIVIDER * (clownmdemu->state->mega_cd.m68k.bus_requested || clownmdemu->state->mega_cd.m68k.reset_held ? 1 : Clown68000_DoCycle(clownmdemu->mcd_m68k, m68k_read_write_callbacks));
    }
    This is the code for running the Mega CD's CPU. Every instruction, it would check if the CPU was idling, and, if it was, then it would tell the rest of the emulator to wait until the next cycle and check again. If the CPU was still idling by then, then it would wait and check again on the cycle after that, and so on, and so on... 480,000,000 times per second.

    Ironically, by doing things this way, it was slower to emulate a CPU that was doing absolutely nothing than one that was actually running. This design flaw was enough to land the function near the top of the list in Visual Studio's profiler.

    [​IMG]

    The solution to this is fairly simple: instead of checking whether the CPU is idled every cycle, only check once when the CPU is synchronised with the rest of the system. This way, until the CPU is synchronised again, the emulator can safely assume that it will remain idle indefinitely. If something were to resume the CPU, then the CPU would have to be synchronised, which would cause the emulator to perform the check again, meaning that the emulator cannot possibly miss when the CPU stops being idle.

    After making this minor change, the function disappears from the profiler! Considering how much CPU time that function took, this change should make the emulator considerably more performant!

    [​IMG]

    Complete Window Plane Support
    The window plane is a niche feature of the Mega Drive's Video Display Processor, which allows sections of the screen to be overlapped by an unmoving border graphic. This is typically used for things like HUDs, such as in Castlevania Bloodlines.

    [​IMG]

    Though I described the window plane as an overlay, the way that it really works is that it is rendered instead of Plane A. This effect is not obvious until transparency is introduced:

    [​IMG]

    In the above image, Plane B draws orange dots, Plane A draws blue dots, and the window plane draws green dots. As can be seen, no blue dots appear where there are green dots.

    Or, at least, that is how it should be. Here is what my emulator was actually rendering:

    [​IMG]

    Horizontally, the window plane simply overlaid Plane A. While this may seem like an unimportant edge-case, I have received complaints before from a Sonic ROM-hacker that relied on this feature working correctly for their hack's debug display.

    Accurately recreating this behaviour required extensive modifications to the Video Display Processor emulation code, as well as the unfortunate removal of a few optimisation that I was rather fond of. In particular, Plane A could no longer draw tiles partially past the edges of the screen, as this conflicted with the window plane needing to be drawn at the edges of the screen instead. However, in the midst of these heavy refactoring, some new optimisations were made, resulting in the code being overall slightly faster than before.

    When all was said and done, the inaccuracy was no more, and the emulator could correctly render the window plane!

    Another noteworthy aspect of this process was the homebrew which was used to test the emulator: I created it, using my C++ Mega Drive toolchain. Compared to writing homebrew from scratch in assembly, writing it in C++ was absurdly easy, with a working prototype being produced in a mere half hour. The ease of use also made it simple to convert the homebrew to render in Interlace Mode 2, allowing me to verify the window plane's behaviour when used in tandem with 8x16 tiles. I wish that programming in assembly was this fast!

    Debugger Improvements
    Developing the aforementioned homebrew had made me aware of a number of deficiencies in the frontend's array of debuggers. These have now been remedied, resulting in a smoother debugging experience!

    [​IMG] [​IMG]
    The VDP registers were rather assorted. Now there is more grouping!

    [​IMG] [​IMG]
    The SSG-EG settings were previously a series of obscure numbers. Now it is clear what they actually are!

    [​IMG] [​IMG]
    The old sprite viewer was gruesome, especially in motion. Now it has some flair to it!

    Also new to the frontend is a VSRAM debugger! For all of this time, the frontend has lacked the ability to view an entire portion of RAM! VSRAM in particular contains the data for vertically scrolling Plane A and Plane B: each value controls the scrolling of a 16-pixel 'slice' of the plane. The data is interlaced so that the first value is the first slice of Plane A, then the next value is the first slice of Plane B, then the next value is the second slice of Plane A, etc.

    [​IMG]
    Sonic the Hedgehog 3 uses VSRAM for its crumbling-ground effect!

    Finally, each portion of RAM has been given a memory viewer. This means that the user now has the option to view VRAM as a series of bytes instead of a series of tiles, and likewise for viewing CRAM as words instead of colours. The VRAM visualiser's 'save to file' button has been moved to the memory viewer, allowing all portions of RAM to be saved to a file for further examination by the user. This can be used to extract a game's tiles, plane map data, colour palettes, audio samples, decompressor output, and other data.

    Correct V-Counter in Interlace Mode 2
    Interlace Mode 2 is a seldom-used feature of the Mega Drive that essentially causes it to switch from displaying at 320x224@60 to 320x448@30. This is useful for adding split-screen to a game, as it enables two normal-resolution screens to be displayed at once, stacked on top of each other, albeit squashed. I am aware of only two games that use the feature for this purpose: Sonic the Hedgehog 2 and Combat Cars.

    For much of my emulator's life, Combat Cars was unable to boot, so I was not able to see its split-screen in action. However, improvements in emulation accuracy have since enabled the game to be played! Though, upon starting two-player mode, the user is subjected to a troubling sight:

    [​IMG] [​IMG]
    Imagine these two frames flickering back and forth, over and over.

    The split-screen is completely broken, with each player's view taking up the whole screen and alternating between each other every frame.

    In order to switch between the first and second players' screens, the game needs to know when the VDP has finished drawing the top half of the screen. Sonic the Hedgehog 2 does this using a horizontal interrupt, but the VDP register debugger shows that this is not what Combat Cars does, leaving only one other option: the V-counter.

    The V-counter operates slightly differently in Interlace Mode 2: the V-counter is only 8-bit, giving a maximum value of 255, however in Interlace Mode 2, the screen can be up to 480 lines tall, which would require 9 bits, so, to work around this limitation, the V-counter's first bit is discarded, and instead used to store the ninth bit. By doing this, the V-counter can represent the full height of the screen, albeit without being able to distinguish even and odd lines.

    My emulator has never replicated this behaviour, which could explain why Combat Cars is malfunctioning. I corrected this, booted the game, and...

    [​IMG]

    ...the split-screen worked! No more flickering!

    With the emulator's 'tall interlace mode 2' option, I could even eliminate the awkward squashing, allowing the split-screen to be seen the way that it was meant to be!

    [​IMG]
    I wonder if the HUD and split-screen were inspired by Sonic, given how similar they are.

    VDP Bug Emulation
    The Mega Drive's GPU - the Video Display Processor - has a number of defects and undocumented behaviours, two of which are shown by the below screenshots:

    [​IMG] [​IMG]
    What is meant to be displayed, and what is displayed.

    The numbers are all supposed to read '0123456789', but the digits immediately to the right of the overlapping square (actually the window plane) read '45' instead of '23'. Additionally, while multiple instances of the number '0123456789' are visible, there is really only one, with an undocumented behaviour causing it to be repeated.

    Previously, my emulator recreated neither of these. Games normally do not invoke these behaviours (in fact, the Mega Drive's development manuals warn developers against this), however, very rarely, some games do, necessitating their emulation!

    The issue with the incorrect tiles being drawn appears to have to do with the first pair of tiles beginning before Plane A's left boundary, technically giving it a negative X coordinate offset. This causes the tiles to fetch not just the wrong graphical data, but also the wrong V-scroll data. The X coordinate offset used to obtain graphical data is rounded up to a positive number, causing it to fetch the data of the tile pair to the right. Meanwhile, the X coordinate offset remains as-is when used to fetch the Y-scroll data. Because the offset is negative, the fetched Y-scroll data is that of a column to the left. If Plane A's left boundary is at the exact left side of the screen, then the negative offset causes the X coordinate to underflow, wrapping around and fetching the Y-scroll of the furthest-right column instead.

    The endlessly-repeated number is caused by setting the plane width to an invalid value; this causes the plane width to be set to 32, but also oddly causes the plane height to become 1. With the plane only being one row tall, it repeats vertically across the screen, causing the number to appear multiple times.

    Closing
    Work on this update began immediately after the release of v0.8 (and v0.8.0.1), and has been going non-stop since then. Every time that I think the update is ready for release, I would find a new feature to add or behaviour to emulate, resulting in an update that is jam-packed with features that would normally be the highlight of any other update. What a way to celebrate v1.0.

    As for future plans, a glaring missing feature is the YM2612's Low Frequency Oscillator, but I need to find a good way of testing it before I can implement it. Finally migrating to SDL3 (or another library entirely, such as GLFW) would be nice too, allowing native multi-window to be enabled by default. I could also look into adding the ability to step through the emulated console one or more cycles at a time, which could be useful for debugging effects which are done by interacting with the VDP mid-way through the screen being drawn. Of course, there is also the improvement of Mega CD emulation; being able to boot the Mega CD version of Sonic Megamix would be a great milestone! Even after three years, there is still so much to do...
     
    Last edited: Dec 18, 2024
  5. Clownacy

    Clownacy Retired Staff lolololo Member

    Joined:
    Aug 15, 2014
    Messages:
    1,065
    (Read this on the blog if the images look wrong.)

    v1.1
    Try it in your web browser: clownmdemu.clownacy.com
    Download: Standalone, libretro

    I was expecting the first update after v1.0 to be small, but it ballooned into something massive; compatibility has been greatly improved in various areas, and many features have been emulated for the first time!

    VDP Behaviour Accuracy
    The Mega Drive's Video Display Processor has many strange, undocumented, and unintuitive behaviours, some of which are so obscure that they are unlikely to be relied upon by any games. Because apparently no games require these behaviours, it is difficult to know that they exist in the first place, but this is where homebrew can be very useful!

    Once again, Nemesis's suite of VDP tests proves to be invaluable, as it thoroughly demonstrates many of these behaviours, allowing developers to easily tell if their emulators correctly recreate them! Additionally, the source code to this homebrew is publicly available, allowing developers such as myself to study the code to see exactly how the homebrew is invoking these quirks.

    Unfortunately, the source code is often light on documentation, with the absolute worst case being that it does not explain that DMA Fill operations to CRAM and VSRAM do not fill with the usual DMA Fill value, but rather an arbitrary value from the VDP's FIFO. I was only able to figure this particular quirk out by conveniently remembering some posts on the SpriteMind forum that I had read long ago. Without that, I may never have understood how the homebrew was doing what it did.

    Regardless, it is still an incredibly useful tool, without which I would not have been able to implement so many of the VDP's edge-cases!

    After several hours of trial-and-error and analysing the source code, I was able to make my emulator pass many more of its tests, going from 49/122 to 108/122!

    [​IMG][​IMG]

    CD-Controller Emulation Accuracy
    There are multiple ways for a Mega CD game to load data from its CD: reading in bulk by using a BIOS call, streaming a word at a time by reading a register, and reading in bulk by using a DMA transfer. Until now, this emulator only supported the first option. Neither Sonic CD nor Sonic Megamix use the other options, so, to test this feature, I created some homebrew.

    Not wanting to write homebrew from scratch in assembly, I extended my Mega Drive C++ toolchain to support creating Mega CD homebrew. In particular, it supports creating "Initialisation Program" ("IP"), and "System Program" ("SP") payloads, which are automatically loaded by the BIOS for execution by the MAIN-CPU and SUB-CPU, respectively. These payloads differ from typical cartridge software in their memory layout and bootstrapping code: IP and SP payloads need not perform hardware initialisation, as this is handled by the BIOS, while SP payloads additionally need a small header which declares a series of callbacks for the BIOS to invoke. With this done, I had a small homebrew Mega CD "game" that loaded and displayed its first CD sector.

    [​IMG]

    At first, this homebrew was tested by running it in Genesis Plus GX. What is notable about this emulator is that it performs low-level emulation (LLE) of the Mega CD's BIOS, making it a reliable way of verifying the BIOS's behaviour, unlike my emulator which skips emulating the BIOS completely by leveraging high-level emulation (HLE). I would have instead verified this homebrew by using a real Mega CD, but without an optical drive emulator (ODE), I would be forced to burn numerous CD-Rs to use with the console, which would be a waste. Alternatively, I could have converted the homebrew to "Mode 1", which would allow it to run from a cartridge instead of a CD while still being able to make use of the Mega CD hardware, but I wanted to exercise my C++ toolchain's ability to produce 'proper' Mega CD software. With the homebrew working in Genesis Plus GX, I then began implementing the missing CD data transfer modes in my emulator.

    Since first adding Mega CD support to my emulator, I have learned more about how the console's BIOS calls and CDC unit work. Much of this information came from Devon, who contributed much to the Mega CD emulation, such as its stub BIOS. This information revealed that much, if not all, of the CDC and BIOS call emulation was wrong, so I discarded the existing code and reimplemented it all from scratch, taking into account quirks such as the 128 kbit CD-ROM FIFO and how the 'ROM' BIOS calls were merely wrappers around the 'CDC' BIOS calls.

    After a round of fixing various bugs in this new implementation, the homebrew was able to run correctly! However, Sonic CD failed to boot. Using my emulator's built-in debuggers, as well as Visual Studio's debugger, I located the game's CD-reading code and examined how it was executing, as well as how the emulator responded to it. By doing this, I was able to identify and fix more bugs in the CDC emulation code, allowing the game to boot.

    With Sonic CD working, my focus shifted to Sonic Megamix; the Mega CD version of this famous ROM-hack has never booted in my emulator before, but, with the experience of reverse-engineering Sonic CD's CD-reading code fresh in my mind, I was ready to dive into Megamix's internals to finally unravel this mystery.

    My examination uncovered one more bug (which had to do with data in the CD-ROM FIFO being overwritten before it could be read) but it also revealed some edge-cases in the CDC/BIOS's behaviour. The reason for this was that Megamix's CD-reading appears to suffer from some bugs: rather than issuing BIOS calls until they succeed, it does so until they fail. This results in the BIOS calls 'CDCREAD' and 'CDCTRN' being issued twice. 'CDCREAD' is responsible for preparing the sector at the end of the CD-ROM FIFO for transfer, and 'CDCTRN' is responsible for performing the transfer. This raises the questions of what should happen if the same sector is prepared for transfer twice, and what should happen when the same sector is transferred twice. Should the second call to 'CDCREAD' re-prepare the sector for transfer and signal success, or should it detect that a sector is already prepared and signal failure? Should the second call to 'CDCTRN' transfer a second copy of the sector and signal success, or should it detect that the sector has already been transferred and signal failure? My emulator was doing the former in both cases, while Megamix showed that it should have been doing the latter.

    With these inaccuracies addressed...

    [​IMG][​IMG]

    ...Sonic Megamix booted, at long last!

    But that was not all: the tech demo 'Sonic for Mega CD', as well as the leaked 'v5.0a' version of Sonic Megamix also booted!

    [​IMG][​IMG]

    Improved CD Audio Track Compatibility
    This emulator is not currently able to play Mega CD games directly from their original discs; instead, users are expected to extract the contents of each of their discs to a file, which the emulator can use. This is a common practice in emulation, with such files being termed 'rips', 'dumps', 'copies', and 'backups'. Over the years, many different formats for Mega CD rips have been devised, which is a problem for emulators, as they need to support an increasingly large number of formats to maximise compatibility with users' libraries of games.

    This problem extends to audio formats as well, as there exist Mega CD game formats which encode the audio tracks in Ogg Vorbis, MP3, and WAV. This is even further complicated by the fact that these audio formats can contain audio that differs wildly from that of an actual CD; in particular, audio can have more or fewer than 2 channels, and use a sample rate that is not 44100Hz.

    Previously, I had assumed that all Mega CD game files would use 44.1kHz stereo audio, and so my emulator supported only playing audio files that met that standard. After all, real CDs can only ever contain audio in that format, so one could reasonably expect all rips of a CD to have its audio in the same format; to increase the channel count or sample rate would be to needlessly waste space by adding redundant data, and reducing the channel count or sample rate would lower the quality of the audio by discarding useful data.

    However, this logic fails to account for homebrew and ROM-hacks: instead of being created by extracting data from a CD, homebrew Mega CD audio files can be sourced from live audio recordings, exported from music software, downloaded from the internet, or even extracted from other games. This means that these audio files can use practically any sample rate and channel count, completely shattering the assumption that they would be 44.1kHz stereo.

    I encountered this issue when trying to play Sonic Winter Adventures, which is a ROM-hack of Sonic 1 that was made by Vladikcomper back in 2013. This game comes with a soundtrack of MP3 files, some of which are 44.1kHz, but others are 32kHz. The result of this was that some songs would not play in the emulator, leaving the game without music on its title screen and upon completion of a level.

    To resolve this problem, I added a resampler (clownresampler) to the emulator's disc-rip-reading library (clowncd). With this, I could make the library convert audio to the expected 44.1kHz sample rate at runtime. I also added some logic to upsample mono audio to stereo. By doing this, my emulator was granted compatibility with audio that uses alternate sample rates as well as audio that used only 1 channel.

    CD-DA Fader Emulation
    The CD-DA music emulation is currently quite basic, lacking features such as fast-forwarding and reverse playback. However, one feature that is no longer missing is the fader, allowing for CD-DA volume control and fading. Curiously, Sonic CD always keeps its CD-DA volume slightly below maximum, so this change will cause Sonic CD to have quieter music. This feature is also used by Sonic Winter Adventures, which fades out its music before boss battles.

    Sega's official documentation for this feature is quite lacking, not explaining exactly how the volume values affect the loudness of the audio, nor how the "change speed" value relates to the volume. There is also an inconsistency between the official documentation and official sample code, where the former claims the fader updates 75 times per second while the latter claims it to be 60 times per second. To solve these problems, I examined Sega's BIOS; a disassembly of the Mega CD BIOS can be found on GitHub, revealing the internals of how fading is done. In reality, it is quite simple: samples are multiplied by the volume and divided by 0x400, and the "change speed" is a value that is added to the volume every time the fader updates. The fader is updated every time the CDD interrupt occurs, which is 75 times per second.

    With these details figured out, emulating the fader took only half an hour or so. As far as emulating features goes, this was one of the easiest. As of this change, I believe that every feature which Sonic Winter Adventures uses is now emulated.

    'Coordinate Conversion' Emulation
    The Mega CD provides some graphical capabilities, one being "coordinate conversion", which creates tiles by sampling other tiles. This is done by taking a pair of coordinates, sampling the pixel at those coordinates, adding a delta to the coordinates, and repeating. This is similar to how texture sampling works in shaders.

    At the lowest level, this effect is achieved with "stamps", which are in the same format as sprites and can be 16x16 or 32x32. These stamps are arranged into a grid, forming the "stamp map".

    Here are the stamps that are used by Sonic CD's special stages:

    [​IMG]

    Here also is a section of its stamp map:

    [​IMG]

    Coordinate conversion works by rasterising an image, forming lines from pixels, and then forming a whole image from those lines. The lines are formed from left to right, and the image is formed from top to bottom. Each line is ascribed metadata by the "trace vector", which details the line's starting sample coordinates and delta; the leftmost pixel of the line is assigned the pixel of the stamp map that is at the location specified by the starting coordinates. The coordinates are then advanced by the delta, and the next pixel is assigned the pixel of the stamp map at the new coordinates. This is repeated until the line is complete, at which point, the next starting coordinates and delta are fetched from the trace vector and a new line is produced. This is repeated until all lines are drawn, at which point the process ends.

    Researching this feature was difficult, due to the official documentation for it being very fragmented: the Mega CD Hardware Manual, Mega CD Outline Manual, and Mega CD Development Manual each describe parts of the coordinate conversion process, some containing unique information, and others containing a reworded version of documentation that can be found in another manual.

    Fortunately, Devon provides a thorough and concise explanation of this feature along with some sample code on GitHub, which was enough for me to mostly understand it.

    I still had some uncertainties about the feature, however, such as what should happen when the 'image buffer V-cell height' and 'image buffer dot height' are mismatched. To answer these questions, I expanded my C++ Mega CD homebrew to showcase the output of the coordinate conversion when supplied with various parameters. This answered the last of my questions, allowing me to finally move onto implementing the feature.

    In the process of implementing coordinate conversion, I added the two debug visualisers that are seen above. The size of the stamp map caused the emulator to lag when its visualiser was enabled, which led me to overhaul the visualisers to use hardware-accelerated rendering, greatly improving their performance.

    In comparison to overhauling the visualisers, emulating the coordinate conversion was easy, due to the feature being thoroughly examined before I began implementing it, allowing me to plan the implementation in advance. Within a couple of hours, the first prototype was complete, allowing Sonic CD's title screen, special stages, and D.A. Garden to appear correctly:

    [​IMG]
    Used for the clouds.

    [​IMG]
    Used for the floor.

    [​IMG]
    Used for the planet.

    With this, the only missing features which are relied on by Sonic CD are BuRAM (save data), CDC transfer durations (FMV audio), and possibly audio track lead-in (music timing).

    New Logo
    I was visiting a retail park near where I live when I came across a Dunkin'. Something about its sign struck me as familiar:

    [​IMG][​IMG]

    Oh dear.

    ClownMDEmu's logo was designed to be as minimalist as possible. Unfortunately, the problem with minimalism is that designs tend to look similar. In this case, by sheer coincidence, Dunkin' had also come up with the idea of reducing a name to four letters, arranging them into a square, and using a font that is made of circles.

    Since I am not interested in getting involved in a trademark dispute (and a million 'haha Dunkin' logo funny' jokes), I have switched the font to a rounded version of the 8x8 font that I made for the emulator's custom Mega CD boot ROM.

    [​IMG][​IMG]

    I would have kept the pixelated font, but I find that the hard corners clash horribly with the rounded design of the rest of the logo.

    YM2612 LFO Emulation
    The final major unimplemented feature of the the Mega Drive's main sound chip is the low frequency oscillator (common referred to as 'the LFO'). This is a fairly-simple waveform that modulates an operator's amplitude (volume) and phase (pitch), allowing for quick-and-easy tremolo and vibrato. Very few of the Mega Drive games that I own make use of this feature, and support for it was removed from the modified SMPS sound drivers that are used by many Sonic games. SMPS provides its own means of achieving vibrato and tremolo which are much more powerful than that offered by the LFO (larger modulation range and custom envelope shaping), which could explain why support for the redundant feature was removed.

    While a lack of games which require LFO was one reason that it took so long for the feature to be emulated, the other reason was that thorough documentation of it is practically non-existent: the official Yamaha documentation of the YM2612 only gives a high-level overview, which is of little help in understanding how to emulate it, and community documentation is equally lacking, with Plutiedev simply paraphrasing the Yamaha documentation, the Mega Drive wiki containing barely any documentation at all, and even the incredible SpritesMind YM2612 thread lacking a proper write-up, only debating a handful of details of its functionality with few concrete answers. Tragically, Nemesis had planned on providing a thorough write-up of the feature, in the same vein as what he had done previously with the envelope and phase generators, but life had gotten in the way.

    This lack of documentation has proven to be a problem for other emulators as well, as Eke-Eke, the maintainer of the Genesis Plus GX emulator, can be seen multiple times in the SpritesMind thread asking for any details about the internals of LFO, as many games exhibit inaccuracies when the feature's behaviour is not recreated exactly.

    Despite all of these obstacles, there was one breakthrough that finally made accurate LFO emulation possible: the Nuked OPN2 project, which is a transcription of the YM3438's circuitry to the C programming language (the YM3438 is the CMOS version of the YM2612). With this, the exact behaviour of the LFO is documented in complete detail (barring any potential transcription errors).

    I was not content with merely copying code directly from Nuked OPN2, so instead I set about implementing LFO it in a way that is optimised and natural to read while still being functionally-identical. This did unfortunately reach a limit, as the YM2612's unconventional logic leads to rounding errors which require equally-unconventional code to recreate. This is especially noticeable with the logic for phase generator modulation: the YM2612 implements this using a pseudo-multiplication made of shifts and an addition, but it discards many bits in the process that a true multiplication would not, producing values that are off by 1. As a result, the code for this is less sightly than I would like it to be:

    Code:
    #if 1
        /* This goofy thing implements a fixed-point multiplication using only shifts and an addition.
           Unfortunately, this particular method is required in order to recreate the rounding errors of a real YM2612,
           which prevents me from replacing it with a real multiplication without sacrificing accuracy. */
        static const cc_u8l lfo_shift_lookup[8][8][2] = {
            {{7, 7}, {7, 7}, {7, 7}, {7, 7}, {7, 7}, {7, 7}, {7, 7}, {7, 7}},
            {{7, 7}, {7, 7}, {7, 7}, {7, 7}, {7, 2}, {7, 2}, {7, 2}, {7, 2}},
            {{7, 7}, {7, 7}, {7, 7}, {7, 2}, {7, 2}, {7, 2}, {1, 7}, {1, 7}},
            {{7, 7}, {7, 7}, {7, 2}, {7, 2}, {1, 7}, {1, 7}, {1, 2}, {1, 2}},
            {{7, 7}, {7, 7}, {7, 2}, {1, 7}, {1, 7}, {1, 7}, {1, 2}, {0, 7}},
            {{7, 7}, {7, 7}, {1, 7}, {1, 2}, {0, 7}, {0, 7}, {0, 2}, {0, 1}},
            {{7, 7}, {7, 7}, {1, 7}, {1, 2}, {0, 7}, {0, 7}, {0, 2}, {0, 1}},
            {{7, 7}, {7, 7}, {1, 7}, {1, 2}, {0, 7}, {0, 7}, {0, 2}, {0, 1}}
        };
    
        const cc_u16f f_number_upper_nybbles = f_number >> 4;
        const cc_u8l* const shifts = lfo_shift_lookup[phase_modulation_sensitivity][phase_modulation_absolute_quadrant];
        step = (f_number_upper_nybbles >> shifts[0]) + (f_number_upper_nybbles >> shifts[1]);
    
        if (phase_modulation_sensitivity > 5)
            step <<= phase_modulation_sensitivity - 5;
    
        step >>= 2;
    #else
        /* This is what the above code is an approximation of. */
        static const cc_u8l lfo_lookup[8][8] = {
            {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00},
            {0x00, 0x00, 0x00, 0x00, 0x01, 0x01, 0x01, 0x01},
            {0x00, 0x00, 0x00, 0x01, 0x01, 0x01, 0x02, 0x02},
            {0x00, 0x00, 0x01, 0x01, 0x02, 0x02, 0x03, 0x03},
            {0x00, 0x00, 0x01, 0x02, 0x02, 0x02, 0x03, 0x04},
            {0x00, 0x00, 0x02, 0x03, 0x04, 0x04, 0x05, 0x06},
            {0x00, 0x00, 0x04, 0x06, 0x08, 0x08, 0x0A, 0x0C},
            {0x00, 0x00, 0x08, 0x0C, 0x10, 0x10, 0x14, 0x18}
        };
    
        step = f_number * lfo_lookup[phase_modulation_sensitivity][phase_modulation_absolute_quadrant] / 0x100;
    #endif
    With all of that done, games are now able to make use of this underutilised feature!



    FM3 Multi-Frequency Operator Order
    A bug that was reported on GitHub was that Another World had some broken sounds. I had recently picked up a copy of this game to test another issue, so I was able to experience this problem for myself. True to the reporter's word, some sounds were completely incorrect, notably the splashing sound that would occur at the start of the game as the protagonist climbs out of a pool. With my emulator's debugger, I could tell that the broken sound made use of the YM2612's per-operator frequency feature, though I had no way of knowing if that had anything to do with the bug.

    To figure out exactly which part of the sound was causing the bug, I recreated the YM2612's configuration in some C++ homebrew, and gradually disabled features until the resulting sound was identical between my emulator and Nuked OPN2. Eventually, I found that the problem stemmed from the frequencies of operators 1 and 2 being swapped, and yet, looking at the source code, I could not see how the operators could possibly be getting swapped. That changed, however, upon looking at documentation:

    [​IMG]

    A9, AA, A8, and then A2...? The ordering is all wrong! Operators 2 and 3 being swapped is normal, but operator 1 coming after those two and operator 4 being before them all? None of that makes any sense! Doubting the documentation, I checked Nuked OPN2 only to be further persuaded - the frequency registers really are in a garbled order!

    Code:
    case 1: /* OP1 */
       chip->pg_fnum = chip->fnum_3ch[1];
       chip->pg_block = chip->block_3ch[1];
       chip->pg_kcode = chip->kcode_3ch[1];
       break;
    case 7: /* OP3 */
       chip->pg_fnum = chip->fnum_3ch[0];
       chip->pg_block = chip->block_3ch[0];
       chip->pg_kcode = chip->kcode_3ch[0];
       break;
    case 13: /* OP2 */
       chip->pg_fnum = chip->fnum_3ch[2];
       chip->pg_block = chip->block_3ch[2];
       chip->pg_kcode = chip->kcode_3ch[2];
       break;
    case 19: /* OP4 */
    default:
       chip->pg_fnum = chip->fnum[(chip->channel + 1) % 6];
       chip->pg_block = chip->block[(chip->channel + 1) % 6];
       chip->pg_kcode = chip->kcode[(chip->channel + 1) % 6];
       break;
    Accounting for this in my emulator finally got Another World's sounds to play correctly.

    Lock-On Technology Save Data
    Immediately after releasing v1.0, which prominently featured support for creating persistent save data, it was reported that Sonic 3 & Knuckles did not save. Being such a major title for the Mega Drive, this defect was a high priority to fix.

    The cause of this problem is that ClownMDEmu only creates save data if the game's ROM header declares that it uses save data. While the emulator could just always create save data by default, that would break compatibility with games which allegedly have anti-piracy measures that are triggered by the presence of save-data functionality; the cartridges for these games lack save data hardware, so the anti-piracy measures are not tripped when using them.

    Sonic 3 & Knuckles uses the ROM header of Sonic & Knuckles, which does not declare save data, however it does appear to contain data which indicates where the Sonic 3 ROM header is, which does declare save data. By using this data to make the emulator search both headers, the emulator finds the save data metadata and enables the game to correctly save its data.

    Migration to SDL3
    It finally happened; SDL3 has been officially released. When support for SDL3 was first added to this emulator's frontend, the library was immature and riddled with bugs. Since then, the situation has greatly improved, with the only bug that I could find apparently actually being a bug in Nvidia’s Linux driver.

    Because of this, I am finally confident in moving the frontend over to SDL3 permanently! With this move comes future-proofing, the elimination of platform-specific code, and...

    [​IMG]

    ...native multi-window! At last!

    Closing
    Yet another giant update to add to the list! I am glad that v0.4 was not the only time that I got to release so many improvements in a single update! Like with that update, however, I am feeling quite burnt-out, so do not expect v1.2 to arrive for a while!

    My mind has been wandering lately to adding support for the Master System and Game Gear. I am not sure what I would be getting myself into by trying to do so, but unlocking another huge library of games for my emulator to run does have a lot of appeal to me...
     
    Last edited: Mar 8, 2025
  6. Red2010 is now

    Red2010 is now A Normal RomHacker with occupations. Member

    Joined:
    Apr 24, 2023
    Messages:
    84
    Location:
    Somewhere in Spain
    It won't be long before we see the end of Lester's story running on this Emulator

    If anyone played Heart Of The Allien (Another World II) they know exactly what happens to the protagonist of the first part halfway through the story
     
  7. Clownacy

    Clownacy Retired Staff lolololo Member

    Joined:
    Aug 15, 2014
    Messages:
    1,065
    (Read this on the blog if the images look wrong.)

    v1.2
    Try it in your web browser: clownmdemu.clownacy.com
    Download: Standalone, libretro

    While not the gigantic leap that v1.1 was, v1.2 is an update that refines and expands various parts of the emulator core, improving compatibility with many games. The flagship feature of this update is the new low-pass filter, which takes audio accuracy to a whole new level!

    YM2612 Timer Improvements
    Immediately after releasing the previous update, some people pointed out that the voice clips in several games were too slow. Notably, these games were powered by the GEMS sound driver. I had known about this problem for quite some time, but was unsure what the cause could be: the voice clips being too slow would suggest that the emulated Z80 is too slow, but, if anything, it should be too fast, due to the lack of DMA transfer duration emulation meaning that the Z80 should have more cycles per frame than it would on a real Mega Drive. Because of this, I was without a lead on what the root of this problem was, and so it remained unsolved.

    That changed when Malachi gave me a very useful tip: GEMS uses the YM2612's Timer A to control the playback of samples. I had never considered this, having only ever seen the playback of samples be controlled by manually counting Z80 cycles. With this in mind, it was clear that something must be wrong with the Timer A emulation!

    It did not take me long to discover that the code for this feature was woefully broken, often failing to update the timer at all. The code also did not implement the YM2612's CSM feature very well, due to not synchronising with the actual sample generation, causing the key-on/off to be latent by many samples.

    Addressing all of these problems made the voice clips in Earthworm Jim (a game that uses GEMS) to sound correct!

    Cartridge Bank-Switching
    After reading about James Groth getting the Titan Overdrive tech demos running in his Mega Drive emulator, I wanted to see how well they would run in mine. I booted up Overdrive 2, only to be greeted with this:

    [​IMG]

    The "SSF2-style bank switching" that it is describing is something that I have read about many times before: it is a way for games to overcome the standard cartridge size limit of 4MiB by dividing the ROM into blocks of 512KiB and making any 8 of them visible to the console at any given time (though, the first block is always hardcoded to block 0). It was allegedly only used by a single game, Super Street Fighter II, which is why it is often referred to as 'the SSF2 mapper'. However, I have heard that the mapper was a generic product offered by Sega to all developers, and not something that was made specifically for Street Fighter, making its nickname something of a misnomer.

    To get Overdrive 2 running in my emulator, I would need to emulate cartridge bank-switching. Fortunately for me, this is incredibly simple: the cartridge mapper's hardware populates part of the 68k's IO region with an array of bytes, each one controlling which bank is assigned to a 512KiB 'slot' in the cartridge region. With that implemented, Overdrive 2 would boot!

    [​IMG]

    Unfortunately, due to being Overdrive 2, my emulator does not stand a chance of emulating any of the demo's fancy tricks properly:

    [​IMG]

    However, as a nice bonus, this work did also get Super Street Fighter II to boot in my emulator for the very first time:

    [​IMG]

    Earthworm Jim Music Speed
    For a very long time, Earthworm Jim's music has played at an unstable speed in ClownMDEmu. I never paid much attention to this problem, as the game was known to have a similar problem on some real Mega Drives.

    At the heart of this problem was that early models of Mega Drive only allowed the YM2612's status information to be read from the first of its four ports. Earthworm Jim's GEMS (v2.5?) sound driver mistakenly tries to read the status information from one of the other ports, causing it to misbehave and make the music stutter. Later Mega Drive models behave differently, allowing the status information to be read from any of the four ports.

    Because my emulator specifically emulates an early Mega Drive, I did not feel the need to "fix" this issue, as it was behaving exactly as intended. Or, at least, I thought it was, until I looked at what my emulator was actually doing:

    Code:
       else if (address >= 0x4000 && address <= 0x4003)
       {
           /* YM2612 */
           /* TODO: Model 1 Mega Drives only do this for 0x4000.
              Accessing other ports will return the old status,
              and only for a short time. See Nuked OPN2 for more details. */
           value = SyncFM(callback_user_data, target_cycle);
       }
    ...It was not emulating the flawed behaviour of early Mega Drives at all: instead, it was recreating the behaviour of later Mega Drive models, which should not be exhibiting the bug in Earthworm Jim.

    This bug first manifested in my emulator when I changed the Z80 CPU's interrupts to not be 'sticky': the Z80 does not remember if an interrupt was previously requested, instead it constantly checks if an interrupt is currently being requested. This means that it is possible for the following to occur:
    • An interrupt is requested.
    • The Z80 is unable to respond to the interrupt until later.
    • On the next scan-line, the interrupt request ends.
    • The Z80 is finally able to respond to the request, but, because the request had ended, it does not do so.

    Because of this, it is possible for interrupts to be 'lost'. Games often use an interrupt to time their music, so when this change caused Earthworm Jim's music to play at the wrong speed, I knew that it must have had to do with lost interrupts.

    Prodding around in my emulator, I found that the lost interrupts coincided with two things: the Z80's bus being requested, and the Z80 explicitly blocking interrupts.

    The latter stems from the clunky design of Earthworm Jim's GEMS sound driver, which unnecessarily blocks against interrupts for large periods of time, leaving the Z80 regularly unable to respond to interrupts.

    As for the issue of the Z80's bus being requested, I figured that it was possible for inaccurate timing emulation to be causing the entire scan-line during which the interrupt is requested to occur during an operation where the Z80's bus is requested for a long time, such as a DMA transfer. However, this was not likely to be the case as my emulator does not currently emulate DMA transfer durations, resulting in them being instantaneous. To my knowledge, there are no other operations that would involve regularly holding the Z80's bus for anything more than a few instructions, leaving me at a dead-end.

    I then studied what the other CPU - the 68k - was doing whenever a lost interrupt occurred, and found that it was consistently in the middle of executing a particular couple of functions:

    Code:
    0024DAA2: move.w  #$100,($A11100).l
    0024DAAA: nop    
    0024DAAC: nop    
    0024DAAE: btst.b  #8,($A11100).l
    0024DAB6: bne.s   $24DAAE
    0024DAB8: lea     ($A10003).l,a0
    0024DABE: bsr.w   $24DAD2
    0024DAC2: move.w  d0,($FFFF7C).l
    0024DAC8: move.w  #0,($A11100).l
    0024DAD0: rts
    Code:
    0024DAD2: lea     ($24DAF6).l,a1
    0024DAD8: move.b  (a1),2(a0)
    0024DADC: clr.l   d0
    0024DADE: moveq   #8,d1
    0024DAE0: move.b  (a1)+,(a0)
    0024DAE2: nop    
    0024DAE4: nop    
    0024DAE6: move.b  (a0),d2
    0024DAE8: and.b   (a1)+,d2
    0024DAEA: beq.w   $24DAF0
    0024DAEE: or.b    d1,d0
    0024DAF0: lsr.b   #1,d1
    0024DAF2: bne.s   $24DAE0
    0024DAF4: rts
    The first function requests the Z80's bus, calls the second function, then releases the bus. The second function reads from the console's controller.

    What is odd about this logic (aside from the bugged, nonsensical 'btst #8' instruction that actually functions as a 'btst #0') is that it goes about reading the controller in an absurdly inefficient way, taking far more time than necessary. This causes the Z80 bus to remain requested for a much longer time than normal, creating a large period where the Z80 is unable to respond to interrupts.

    This code is ran immediately after the 68k responds to an interrupt. If the 68k were able to respond to the interrupt faster than the Z80, then theoretically it could have time to execute the above code, requesting the Z80's bus and preventing it from responding to the interrupt long enough for the interrupt request to end, causing the Z80 to miss the interrupt entirely.

    With this in mind, the problem appeared to be that the 68k was somehow faster than it should be, being able to reach the slow controller-reading code before the Z80 has a chance to respond to the interrupt. So then what could it be that the 68k is not doing slowly enough? I mentioned earlier that DMA transfer durations are not currently emulated, but those are unrelated to this as no DMA transfers occur before the controller code is executed. The only thing that occurs before the controller code is the interrupt itself.

    It was upon observing this that I remembered that neither of my CPU interpreters emulate how long it takes for each CPU to respond to an interrupt. This is partly because of a lack of test coverage, as the test suites that I use only test the timing of instructions, not interrupts. I eventually found some documentation online which explained that the 68k takes roughly 44 cycles, and that the Z80 takes roughly 13. Could this slight delay be enough to give the Z80 time to respond to the interrupt before its bus is seised by the 68k?

    Indeed it was: I compiled and launched my emulator, and was finally able to hear Earthworm Jim's music at its proper speed once more! There is still some slight instability, but this should eventually disappear as more 68k-impeding behaviours, such as Z80 cycle-stealing, are implemented in the future.

    YM2612 SSG-EG Inaccuracy
    As I was reading the ever-useful audio issues list on GitHub, I noticed that a homebrew game, Tanglewood, was listed as making heavy use of the seldom-used SSG-EG feature. Being a hobbyist game, I was curious of if it was free to download, as it would make for a quick and easy way to test SSG-EG. It is not; the game is sold commercially. However, a free demo is available.

    After downloading the demo, I was able to access its sound test and listen to its various music tracks and sounds. Looking at my emulator's debugger, I could see that several songs and sounds did indeed make use of SSG-EG. To my surprise, a few sounds appeared to be broken: sound 1C sounded very different to how it did when played by Nuked OPN2, and sound 43 had an unpleasant buzzing noise.

    After spending far too long studying Nuked OPN2's code to see what it was doing differently, I noticed that SSG-EG is meant to be applied per-operator, but my emulator was applying it per-channel. This meant that all operators were being forced to use the same SSG-EG configuration, when normally they are able to use their own unique configurations.

    After correcting this mistake, Tanglewood began to sound identical to how it does with Nuked OPN2.

    Plane Debugger Background Colour
    Ever since their introduction, the plane debuggers have had an odd quirk: some graphics would appear with a border around them, despite this border not existing in the emulated game.

    [​IMG]

    The reason for this is that transparent pixels were not being treated as transparent at all, and instead displaying the first colour entry in their respective palette line.

    [​IMG]

    This became a problem for MDTravis, a user who was trying to use the plane debugger to create accurate maps of levels in Alex Kidd, and had to constantly edit-out the borders.

    To rectify this, I changed the debuggers to display transparent pixels as the designated 'background colour' instead.

    [​IMG]

    And, with that, the problem is solved!

    Vastly-Improved Audio Low-Pass Filter
    A low-pass filter was added to ClownMDEmu all the way back in July 2022. However, this filter was a so-called 'brick-wall' filter, which completely erased frequencies above its threshold while perfectly preserving ones below it. This is not accurate to how a real Mega Drive works, causing the audio to sound slightly different.

    Real Mega Drives use a very rudimentary low-pass filter, not achieved with discrete logic chips but rather with analogue electronic components. This is what I had learned from James Groth's blog, however, that blog merely paraphrases a post from the Sega-16 forum, which provides no proof of its own. Not wanting to go out of my way to implement an entire feature based on mere hearsay, I opted to reverse-engineer this electronic filter myself.

    In his blog post, Groth mentioned that one of the Mega CD's official service manuals contained a schematic of the console, including its low-pass filters, so it was my hope that one of the Mega Drive's service manuals would contain a similar schematic. Fortunately, I was able to find two such manuals, specifically ones for the "Model 1" Mega Drive, which is renowned for its audio quality. Both manuals can be found here, on Nemesis's website.

    After looking around for a few hours, I was able to find what appeared to be the low-pass filter circuitry:

    [​IMG]

    • Top-left line: The SN76489's audio channel (input).
    • Bottom-left line: The YM2612's left audio channel (input).
    • Top-right line: The AV port (output).
    • Bottom-right line: The headphone jack (output).
    This appears to be a passive RC filter, so-called because its only components are a resistor and a capacitor. This implements a first-order low-pass filter, which attenuates the signal by 20 decibels for each tenfold increase in frequency past a given threshold. This threshold is determined by the resistor's resistance and capacitor's capacitance, and can be computed with the formula '1 / (2 * pi * resistance * capacitance)' (resistance is measured in Ohms, capacitance is measures in Farads).

    Because the resistor is 10 kiloohms and the capacitor is 5600 picofarads, this formula becomes '1 / (2 * pi * (10 * (10 ^ 3)) * (5600 * (10 ^ -12)))', which produces 2842Hz as its result. This is known to be the cut-off frequency of Mega Drive revisions VA3 to VA6.8. The VA0-VA2 revisions allegedly use a higher frequency, but, without any schematics to examine, I cannot confirm this.

    With the frequency now determined, I could set about implementing a first-order low-pass filter in my emulator. As Groth explains in his blog post, this can be implemented extremely efficiently using an IIR filter, which requires some coefficients that first need to be computed. I used this website to do so, inputting 2842 as the cut-off frequency, 53267 (the YM2612's sample rate) as the sampling frequency, and 1 as the order, which gave me the coefficients 4.910 and 6.910. Samples can be filtered using these coefficients with the following algorithm:

    Code:
    output = (sample + previous_sample + previous_output * 4.910) / 6.910
    previous_sample = sample
    previous_output = output
    Groth warns that one should be careful to avoid subnormal floating-point numbers, as they will adversely affect the filter's performance. Since I do not want floating-point logic in my emulator anyway, I decided that the solution to this problem would be to convert the filter to use only integers. This is straightforward enough, and can be accomplished with the following modification to the algorithm:

    Code:
    output = ((sample + previous_sample) * 100 + previous_output * 491) / 691
    previous_sample = sample
    previous_output = output
    As an added optimisation, the filter can also be converted into a fixed-point multiplication by multiplying each number by 0x10000 and dividing it by 691 (ideally with rounding):

    Code:
    output = ((sample + previous_sample) * 0x250C + previous_output * 0xB5E8) / 0x10000
    previous_sample = sample
    previous_output = output
    This reduces the filter to a simple pair of multiplications and a bit-shift. No floating-point nonsense here!

    The filter needs to be applied to the YM2612's left and right channels individually, and a separate filter with different coefficients is needed for the SN76489 (due to its different sample rate). With that done, my emulator's new low-pass filter was complete!

    As mentioned previously, this new filter attenuates gradually, whereas the old one was all-or-nothing. This difference is extremely noticeable, as it causes treble and bass to wildly differ in volume relative to each other. This quietens the harsh, grating higher frequencies and boosts the smooth lower frequencies, greatly affecting the volume balancing of music. Many songs that were previously unpleasant to listen in my emulator have become much more enjoyable!



    Closing
    It is not often that I find myself releasing an update that mostly consists of bugfixes and accuracy improvements, but it does feel very good to stomp-out some of the remaining discrepancies between my emulator and an actual Mega Drive, as it shows that the emulator is finally reaching some degree of maturity. At one point, ClownMDEmu was a barely-functional, Linux-only program with no user interface that only correctly ran a handful of games, but now it runs on various platforms (including web browsers) with a fleshed-out user interface and can run practically any Mega Drive game that is thrown at it.

    This update did arrive sooner than I expected, but only because I expected the update after v1.1 to contain some big features, like Master System emulation. Perhaps, instead of emulating yet another console, I should turn my attention back to the Mega CD. Unfortunately, despite owning a Mega CD, I am not very familiar with Mega CD games, making it a difficult add-on to test my emulation of.
     
    Last edited: Mar 8, 2025
  8. Clownacy

    Clownacy Retired Staff lolololo Member

    Joined:
    Aug 15, 2014
    Messages:
    1,065
    v1.2.1
    Try it in your web browser: clownmdemu.clownacy.com
    Download: Standalone, libretro

    Just a small update to fix a crash that would happen when minimising the Tiles debugger, and also to improve the accuracy of 68000 exceptions. The latter fixes VS Puyo Puyo Sun (patched) crashing after its Sega screen.