Development: MMedia Replacement

MMedia Replacement

MMedia has a awkward API and other issues... (more to come...)

So I (RN) Have been coding a replacement for a while (about six months)... Already plenty of code that worked on Windows and Mac (and OSS on unix is pretty easy - which is why I'm putting it off).

RN - "I just need a good API - the platform specific side is EASY compared to this!"

Current API Ideas

Audio

RN - A lot of thought has gone into this! I've spent the last few months dabbling with this.

1. Enumerate Devices (puts all the devices in the system into the AudioDevices array) -
   • wxAudio::Enumerate(AudioDevices)
2. Choose a Device
   • wxAudioDevice MyDevice = AudioDevices[x];
3. Open the Device
   • MyDevice->Open(pMyEvtHandler);
4. Create a Stream (Assuming output)
   • wxAudioOutputStream* pAudioStream = pMyDevice->CreateStream();
     • Output only? Does DirectSound create multiple input (DirectSoundCaptureBuffer) streams?
5. Put some stuff in the stream (or vice versa in input)
   • pAudioStream->Write(pBuffer, nSize);
6. Sound will play... when done will call the wxEVT_AUDIO_REFILL event (example of how to do this) -

   • void OnAudioRefill(wxAudioEvent& Event) { Event.Stream->Write(pNextBuffer, nSize); }

7. When done playing either
   • delete the stream (if just done with the stream) and delete pAudioStream
   • or close the device (will delete all child streams....): MyDevice.Close()

• API Notes - H/W means native API supports it, otherwise it's handled in C++ by wxAudio
  • DirectSound: H/W Mixing & Spatialization
  • KernelStreaming: S/W Mixing & Spatialization
  • AudioUnits(OSX): H/W Mixing & Spatialization
  • OSS (Unix): H/W Mixing, S/W Spatialization
  • Sound Manager (Classic/Carbon Mac): H/W Mixing, S/W Spatialization

Video / MultiMedia Side

Some issues need to be worked out here.

• Enumerate video cards?
  • Well, just one issue it looks like - otherwise I want to use an API like one described for the audio part.

Outstanding Issues

Using Write() to a stream? Would an object that encapsulated media content and then overloading shift operators be more OO? mystream << mywav1 << myau3; Of course, audio content objects can be initialized by a factory that recognizes the content, so the user doesn't need to know the difference between AU and PCM if they don't care to. Not all platforms would support all codecs of course. Maybe I am way off understanding this? (will_varfar lives in #wxwidgets)

Why Duplicate Work?

IMO, there's no reason to worry about this. Portaudio is an elegant, low-latency, easy-to-use, cross-platform audio API that should work great with any toolkit, including wxWidgets. It even resembles the API design you've described. I've used it personally and I've been quite impressed - my program compiled with no changes on Win32, Linux, and Mac OS X. Another interesting implementation to build on is RtAudio

• Agreed. Portaudio is a great and tested library. Maybe we should encapsulate it inside a wxSomething class? -- ebf
  • What about the video player? Wasn't it also a goal for this project?
    • I just want to say that if you start from scratch with another library, you should name it wx<library name>. And that the mmedia docs had very little about video, so this would be more of something like wxAudio. I would ask Mr. Norton to collaborate.

Dont duplicate, do it the wx way

I think anyone working on building native applications, needs to have the capacity to have a seamless wx interface into the audio aspects provided by the os/hardware. I also think as applications and frameworks develop more emphasis will be towards capabilities like tts and speech recognition and 3ds. I see a number of wx applications working towards this direction and it seems a shame they cannot be supported by a wxAudio of some type. If only each layer could build on the good work of the previous... Portaudio, RealTime audio, OpenAL and SDL all provide a level of audio support, from my perspective RealTime audio was the most developed, but none provide the 'wx consistency' in the interface, ALA the wx way.