clownmdemu - The Greatest Mega Drive Emulator Ever (Someday)

Discussion in 'Showroom' started by Clownacy, Jun 23, 2022.

  1. Clownacy

    Clownacy Retired Staff lolololo Member

    Joined:
    Aug 15, 2014
    Messages:
    1,016
    Since September last year, I've been working on-and-off on a Mega Drive emulator. It's not even close to being complete, but when will it ever be? So I figure that I might as well release it now, because there will always be some feature left to be implemented, or some game that doesn't work right. Waiting for it to be 'ready' is a fool's errand.

    clownmdemu!
    You'll never guess what the name's short for! :‎D

    [​IMG]

    [​IMG]




    I haven't done a whole lot of testing with this, but it does appear to work with Sonic 1, 2, 3, & Knuckles, Puyo Puyo, and ROM-hacks like Sonic 2 Recreation. Sonic 3 is a little glitchy at the moment.

    The basic hardware of the Mega Drive is emulated, but not to completion: things like the YM2612's SSG-EG and LFO are missing, as well as support for the VDP's Window Plane and the 68k's instruction cycle durations. Basically, games that do run in the emulator may be missing certain effects. Other games just don't boot, like Combat Cars and Micro Machines. If you want to see exactly which features are and aren't currently emulated, there's a list here.

    As you can see, the emulator comes with some debugging utilities. I figure that they'll come in handy for ROM-hack development, or even just finding out how a game works internally. For example, did you know that Sonic 3's Data Select menu uses Plane B for the foreground and Plane A for the background, rather than the other way around like it usually is?

    Why
    I've been programming for the Mega Drive since late 2012. That's almost 10 years ago! I know practically everything there is to know about how games use the Mega Drive hardware, which means that I know everything that would have to be done in order to run those games on other platforms.

    I figured that writing an emulator would be a good way to put that knowledge to the test.

    Code
    Unlike some other Mega Drive emulators, this one has been created entirely from scratch: no MAME or Gens code here! In fact, I think that the codebase is what makes this emulator unique: it's written in...
    • Rust?
      • No, that's gross and bad and you should be ashamed of yourself for suggesting it. >:‎((((
    • C++20?
      • Not that overcomplicated thing!
    • Go? JavaScript? D? Python? Swift?
      • Get out of my thread.
    • C89...?
      • That's the one!
    ...You might be thinking 'Hey, that's not unique at all!', but here's the thing: my emulator is written in portable C89. What's the difference? Well, many other C projects make mistakes like using fixed-size integer types such as 'uint32_t' for no reason whatsoever (they aren't even guaranteed by the C standard to exist, breaking compatibility with platforms where they don't), treating 'int' like it's always a 32-bit type (breaking compatibility with platforms where 'int' isn't 32-bit), and using logic that only works on little-endian architectures (breaking compatibility with platforms with big-endian CPUs). These projects will only work on certain platforms, while my emulator should theoretically run on any platform that you can compile C for, so long as the RAM requirements are met. Additionally, being written in strict C89 means that the emulator can be built with vintage compilers for ancient platforms (16-bit DOS port, anyone?).

    Another novel feature is that the emulator is separated into two components: the core and the frontend. The core contains all of the emulation logic, while the frontend contains all of the platform-dependent code for reading input and presenting the video and audio to the user. I intend to eventually leverage this to create a libretro core.

    Another major feature of the emulator's code is that it avoids global state: all of the emulator's subsystems access their state through a struct pointer which is passed as a parameter to every function. This is essentially 'proto-C++' object-oriented C. On top of allowing multiple states to be used (thus theoretically allowing multiple Mega Drives to be emulated at once), this also makes for incredibly efficient rewind support. In fact, both rewinding and save states are entirely features of the frontend, as they have been abstracted away from the emulation core itself completely.

    You can read more about the quirks of clownmdemu's code in its README.

    Goal
    My goal with clownmdemu is not to create the most accurate Mega Drive emulator, nor is it to create the fastest. Rather, mine aims for an in-between: to make an emulator that produces 'correct' behaviour externally while being as efficient as possible internally. External accuracy: yes - internal accuracy: no. An example of this is how the FM and PSG are updated: rather than do so once every clock tick as a real Mega Drive would, the emulator only updates the FM and PSG when the 68k or Z80 attempt to access them, doing so in bulk until they have 'caught up' with the rest of the system.

    Download
    But what's the use in describing the emulator here? You can try it for yourself:
    The standalone EXE is 32-bit, and should work on versions of Windows as far back as Windows XP.

    If you're compiling it yourself on Linux, then that should be simple enough: the build script is CMake, and the standalone frontend depends on the FreeType and SDL2 libraries. You can find the latest source code here:
    Usage
    Keyboard controls are remappable, but the controller layout is currently hardcoded. You can find a list of default controls in the emulator's README. Notably, you can quick-save with F5, quick-load with F9, fast-forward with the space key, and rewind with the R key.

    Development
    Since starting this emulator, I've been documenting its development on my blog. So far there have been 8 posts, which I'll list here:
    I'll try to cross-post future posts here, so that anyone who's interested in this emulator can stay up-to-date with its development.
     
    Last edited: Feb 25, 2024
    Pacca, DanielHall, Lone Devil and 9 others like this.
  2. Devon

    Devon Down you're going... down you're going... Member

    Joined:
    Aug 26, 2013
    Messages:
    1,372
    Location:
    your mom
    Pretty cool stuff here. I don't have much time to really look too deeply in it, but a couple things I wanna note with a couple of ROMs I did test:
    • Dual PCM FlexEd does not seem to work properly at all. I have no idea if you are emulating the undocumented Z80 instructions or not, or if there's some kind of other issue.
    • Audio pitch seems to be a tad bit too high to my ears. PSG reminds me a bit of Regen's pitch, which is also off.
    • With my Grimoire of Alice in Fractal Sound ROM, the driver freezes up. I can still move my sound selection and go into the different viewing modes, but I can't do anything else. Might have something to do with Dual PCM?
    • Definitely a result of Dual PCM, but FM and DAC do not function in Dr. Yundong.
     
    Last edited: Jun 23, 2022
  3. Clownacy

    Clownacy Retired Staff lolololo Member

    Joined:
    Aug 15, 2014
    Messages:
    1,016
    Thanks for testing it out!

    The undocumented instructions are already implemented, but a few of the regular instructions aren't, like DAA and CPIR. Dual PCM FlexEd might be falling victim to one of those, or maybe it's just an undiscovered bug in the Z80 interpreter.

    The pitch could be a couple of things... the audio doesn't run at the exact same sample rate as on a real Mega Drive: it tries to account for rounding error introduced by the framerate manager which might be throwing it off. The PSG, I was suspicious could be incorrect: I'm not sure whether the internal timer should check if it's reached zero before or after decrementing. Though, I'm not sure if that makes an audible difference. Now that I have my real Mega Drive set back up, I'll have to do some comparisons.
     
  4. Devon

    Devon Down you're going... down you're going... Member

    Joined:
    Aug 26, 2013
    Messages:
    1,372
    Location:
    your mom
    A quick look at Dual PCM's source code shows that it doesn't use either of them. Are there any other unimplemented instructions?
     
  5. Clownacy

    Clownacy Retired Staff lolololo Member

    Joined:
    Aug 15, 2014
    Messages:
    1,016
    The complete list is DAA, HALT, OUT, IN, RETN, RETI, IM (though this should safely no-op), RRD, RLD, CPI, CPD, CPIR, CPDR, INI, IND, INIR, INDR, OTDI, OUTD, OTIR, and OTDR. Interrupt Modes 0 and 2 aren't implemented either.
     
  6. Devon

    Devon Down you're going... down you're going... Member

    Joined:
    Aug 26, 2013
    Messages:
    1,372
    Location:
    your mom
    Dual PCM uses RETI in its V-BLANK interrupt. IM is probably no big deal, since it sets it to mode 1. On the outside, RETI is basically just RET, but with no condition support. Internally in the Z80, it's designed to release the Z80 from an interrupt state (this is simplifying it, I know), and to be honest, for a Genesis emulator of your scope, it's probably not worth going much into. RETN is designed for returning from a nonmaskable interrupt, which I would say is also outside the scope of this project. You could just probably get away with just treating them both as RET instructions without condition support and call it a day. I wouldn't know if there's some kind of weird obscure thing you can do with them that's possible on Genesis, though.

    EDIT: Unfortunately, changing the RETI to a RET in Dual PCM did not fix it.
     
    Last edited: Jun 23, 2022
    MarkeyJester likes this.
  7. Lone Devil

    Lone Devil Internet War Veteran Member

    Joined:
    Dec 29, 2010
    Messages:
    186
    Location:
    Dallas Texas
    20 seconds in and I can already see myself using this over Regen for its superior VRAM viewer as my upcoming hack rans very well, my only grip is the lack of keyboard rebinding, much prefer arrow keys for movement, ASD for A/B/C, and tab for fastforward.
     
    TheInvisibleSun likes this.
  8. Mohamad20ZX

    Mohamad20ZX Newcomer Trialist

    Joined:
    Oct 25, 2021
    Messages:
    7
    I hope this emulator come out it looks interesting to try out and figure how to make very great ROM hacks
     
  9. SpeedyGonzales

    SpeedyGonzales Newcomer Trialist

    Joined:
    Jul 5, 2022
    Messages:
    6
    Will try this on Windows XP later and it might replace gens on it.
     
  10. Clownacy

    Clownacy Retired Staff lolololo Member

    Joined:
    Aug 15, 2014
    Messages:
    1,016
    It has already come out: there's a download link in the first post.
     
  11. nineko

    nineko I am the Holy Cat Member

    Joined:
    Mar 24, 2008
    Messages:
    1,902
    Location:
    italy
    Isn't that what everyone prefers?
     
    Dark Shamil Khan likes this.
  12. Clownacy

    Clownacy Retired Staff lolololo Member

    Joined:
    Aug 15, 2014
    Messages:
    1,016
    Last edited: Oct 2, 2022
    DeltaWooloo and JGamer2151 like this.
  13. Clownacy

    Clownacy Retired Staff lolololo Member

    Joined:
    Aug 15, 2014
    Messages:
    1,016
    [Cross-post from my groovy blog ;)]

    It's been too long, but finally my emulator has an update!

    Since the first release, the emulator has been greatly optimised, some inaccuracies in the 68000 interpreter have been addressed, and the occasional missing CPU instruction has been added. Compatibility with games should be a bit better than before, but still not great as many essential features of the Mega Drive are not emulated.

    The standalone frontend has had some extra debug menus added, which allow you to view the registers of the YM2612, 68000, and Z80:

    [​IMG]

    New to the emulator is a libretro core frontend, allowing the emulator to be used by libretro implementations such as RetroArch. It lacks the debug menus of the standalone frontend, but makes up for it with features that libretro cores get for free, like customisable controllers and shaders:

    [​IMG]

    In theory, the libretro core should provide a simple way of getting this emulator running on a variety of platforms: just compile the core into a library (static or shared), and use it in tandem with a libretro frontend such as RetroArch.

    During the development of this update, I have set up a test suite for the 68000 interpreter which allows me to check that each instruction does as it is supposed to. It was this test suite that notified me of how the word-size ADDA, SUBA, and CMPA instructions were pitifully broken. I'm surprised that this didn't break Sonic 1, 2, or 3&K, but it did break Linux.

    Yes, Linux: to test my 68000 interpreter, I extracted it from this emulator and used it to create an emulator for the 68 Katy, a hand-built computer that runs Linux. You can read more about it here.

    I also made a small benchmarking tool which measures the speed of the core emulation logic. This is useful for measuring the impact of optimisations and the difference in speed between platforms.

    Overall, this has been a rather incremental update. Rather than being focussed on optimisation and refactoring, I hope that the next update will be focussed on improving compatibility and emulating more features of the Mega Drive.

    You can download the standalone frontend here, and the libretro frontend here.
     
    Last edited: Feb 10, 2023
  14. DashTurbo780

    DashTurbo780 Newcomer Trialist

    Joined:
    Dec 31, 2015
    Messages:
    21
    Location:
    Pittsburgh, PA
    Looks pretty impressive, I'll definitely give it a try.
     
    Clownacy likes this.
  15. JGamer2151

    JGamer2151 Well-Known Member/Lurker Member

    Joined:
    Dec 1, 2020
    Messages:
    94
    Cool update. I do hope that some more optimizations will be made soon along with other additions, since the emulator was designed to balance the line between accuracy and performance (speed). Granted, this isn’t the most fully-accurate nor speed-efficient Mega Drive emulator out there (cough genplusgx cough), but it should hopefully be a middle ground for accuracy and speedups in a built-from-scratch Mega Drive emulator.

    I’m interested to see where this goes.
     
  16. Pacca

    Pacca Having an online identity crisis since 2019 Member

    Joined:
    Jul 5, 2014
    Messages:
    1,175
    Location:
    Limbo
    I really love what your doing here! Emulator blogs are a favorite of mine (I've read the entirety of the dolphin blogs, as well as some others like mGBA), always fun to see a brief overview of the systems features and how you are tackling them, and all the little nuances along the way. And I like the novel ways you're doing it too; I'm excited to checkout all the resources your listing to help familiarize myself with features I've yet to work with, and your reuse of the 68k interpreter for that little linux computer emulator was a fun read :3

    I also really like that one of your goals is to potentially eclipse Regen; I use Bizhawk as my primary emu now, but I still have to come back to Regen from time to time for its' 68K debugger and VDP viewer. It's borked SRAM support makes testing some games in it harder then it should be, the 68K debugger can occasionally be kinda questionable at times, and the emulator has decayed rather poorly as Windows marches on ahead of it. Just recently recommended Regen to someone new to the scene for VDP viewing, and they had substantial problems just setting it up (apparently remapping controls just crashes/freezes in modern Windows, I have my old ini file in the cloud so I never noticed it myself).

    Excited to see where you take it!
     
    Last edited: Feb 15, 2023
    Clownacy likes this.
  17. Dark Shamil Khan

    Dark Shamil Khan TASer lol Member

    Joined:
    Nov 7, 2021
    Messages:
    88
    Location:
    Pakistan
    Interestingly, the SHC splash screen doesn't show up on the emulator properly for some reason. It just shows a black screen. Well that was a few months ago and I haven't tested the new version yet. But have you looked into that tho.

    Anyhow, looks nice. And I loved using it. Especially looking at the vram thingy.
     
    Clownacy likes this.
  18. Clownacy

    Clownacy Retired Staff lolololo Member

    Joined:
    Aug 15, 2014
    Messages:
    1,016
    [Cross-posted from my blog.]

    Keyboard Rebinding

    One shortcoming of the standalone frontend is that it lacks keyboard rebinding: the W, A, S, and D keys will always control the Control Pad's D-Pad, and so on.

    But not anymore!

    [​IMG]

    New to the frontend is full keyboard rebinding! In addition, the default key bindings have been switched to the more common arrow keys and Z/X/C keys combination.

    Unlike some other emulators, this system allows the user to bind multiple keys to the same action: for instance, if the user wanted to bind both the 'Z' key and the 'space' key to the Control Pad's 'A' button, then they can do so!

    It would be pretty frustrating for binding customisations to be lost whenever the program is closed, so support has been added for persistent configuration: settings are saved to a file called 'clownmdemu-frontend.ini', allowing settings such as the keyboard bindings, console region, and V-sync to be remembered by the emulator.

    Options Menu

    Previously, the options would all be managed through the menu bar, but this is quite clunky as the menu bar would close after each option is toggled. To improve the user experience, the options have now been moved to a dedicated menu:

    [​IMG]

    This menu provides a much more intuitive way to change options! Additionally, each option shows a tooltip when hovered over with the mouse, allowing unfamiliar users to understand what they do!

    [​IMG]

    Default Window Sizes

    Another improvement to the user experience is that windows are now given a sane default size, meaning that they will now have a proper size when opened for the first time.

    [​IMG]
    This is dumb.

    [​IMG]
    Much better.

    Recent Software

    Opening the same ROM file over and over again is tedious, so now the emulator keeps a list of the 10 most recent files used:

    [​IMG]

    FM and PSG Debugging Toggles

    The standalone frontend has had the ability to disable individual VDP planes for ages, but now it can also toggle FM and PSG channels. A dedicated menu has been added for this:



    This feature is also available in the libretro core:

    [​IMG]

    PSG Debugger Overhaul

    The PSG debugging menu was butt-ugly before, and has been given a makeover:

    Before:
    [​IMG]

    After:
    [​IMG]

    Support for Alternate PAL Detection Method

    Previously, when playing Sonic the Hedgehog 2 with the emulated Mega Drive in PAL mode, the music would play at slightly slower speed, just like it does in the first game. This shouldn't happen.

    The reason that this was occurring was that the game relies on an alternative method of detecting the PAL video mode: by checking bit 0 of the VDP's control port. This bit should reflect whether PAL mode is enabled or not. Now that this is the case, the game properly detects and accounts for the speed difference in its music, allowing it to play at the proper speed.

    Download
    v0.3 of the standalone frontend can be downloaded here: https://github.com/Clownacy/clownmdemu-frontend/releases/tag/v0.3

    v0.1.1 of the libretro core can be downloaded here: https://github.com/Clownacy/clownmdemu-libretro/releases/tag/v0.1.1
     
    Stdh, EddyTF, Pacca and 3 others like this.
  19. JGamer2151

    JGamer2151 Well-Known Member/Lurker Member

    Joined:
    Dec 1, 2020
    Messages:
    94
    It just gets better and more usable every week, slowly but surely. Great stuff on usability improvements and other features!
     
    Clownacy likes this.
  20. RealMalachi

    RealMalachi You look like an atheist seeing an angel... Member

    Joined:
    Oct 16, 2022
    Messages:
    15
    Location:
    Australia
    This is just incredible. I really dig the UI, and the debugging tools so far are amazing. I can see myself using this emulator a lot for debugging, especially when SRAM and more VDP-related quirks get implemented
    I'll eventually document some unique incompatible features I ran into, but for now here's some questions

    Questions:
    Do you have any interest in putting the libretro core onto RetroArchs online updater, once it becomes more capable?

    How will you/have you handled broken instructions that got fixed on certain revisions? (68k's TAS on non-registers, for example)

    Do you have any plans for interrupt and screen blank debugging? Though not expecting NES emulator quality, since it has more CPUs to keep track of (especially when you add the MCD or god forbid the 32x)
     
    Clownacy likes this.