Sound

[lgromanowski]

According to the roadmap no one is actively working on sound implementation. And no, I won't take it either.
But what I am planning to do is to implement the sound related script features. These would just print some text to std::cout for now. Who ever is gong to implement the sound, will just have to fill in the respective functions.
Also, once we have a console, that works for everybody, this should make testing the sound implementation much easier.

What are the technical requirements for sound? What library (or libraries) are being used?

We are planning to use OpenAL. I think it was already used in the D version, but this part has not been ported yet. Obviously, before anyone can work on the sound implementation, we need to integrate OpenAL in the cmake build system, which at least would require finding or writing a FindOpenAL.cmake file and do some test building.

...which at least would require finding or writing a FindOpenAL.cmake file and do some test building.

FindOpenAL is included in CMake 2.8. CMake is generating valid makefiles with findpackage(OpenAL).

What exactly should I test? linking the libraries against some code?

FindOpenAL is included in CMake 2.8. CMake is generating valid makefiles with findpackage(OpenAL).

It is? Well, then the whole point is moot.

Edit Wait. Here is the roadmap entry

CMake files for OpenAL and decoder libraries

Decoder libraries ... hmm. No idea, what is needed here. Never worked with OpenAL before.

I think I'll take this one. The sound code is going to need a fair bit of restructuring from the old codebase anyway and I think I have a good plan for how to do it.

The 'decoder' libraries are the ones that read wavs, mp3s etc. OpenAL doesn't do that, you have to feed it raw audio data through the API. In a much earlier version we used Audiere for both input and output, but later we switched to OpenAL for output + ffmpeg for input, mostly because OpenAL supports 3D sound natively.

However, ffmpeg is a huge messy library and a real b*tch to compile on Windows IIRC. I am considering switching back to Audiere for input (reading mp3s and such) while keeping OpenAL for output. I've tested this combination before and it seems to work well. I'm going to try it out at least. Audiere will probably need to be included in full in extern/ just like caelum.

A much simpler solution would be FMOD or IrrKlang, but I don't think they jibe very well with the GPL license...

I don't actually know what I am talking about here, but have you considered gstreamer? I know icculus recommends it.

Haven't worked with gstreamer yet, but considering that it can do audio as well as video it wouldn't be a bad choice.

Gstreamer looks interesting, will definitely investigate. Not sure if it handles 3D sound output, but as a decoder for OpenAL and possibly for video output it could be very good.

A side note on video: I've found another project (GemRB, an engine reimp for games like Baldurs gate) that seem to have some Bink decoding code. Not sure about video compatibility, legality/patent issues or feasibility of connecting it to gstreamer or Ogre, but it's worth a look.

I'd only ever really been aware of GStreamer as a GNOME thing, but I do note it has Windows binaries (albeit unofficial). Problem is, it seems that you can't just use one library and be done with it?

I see we are finally making progress with sound.

btw. Nico, you can take the comment about -> script syntax not being implemented out of the soundmanager.hpp file. That was implemented ages ago. I just forgot to remove the comment.

Had a bug that prevented local scripts from running. It is now fixed in my fork. If you run OpenMW with start set to the prisonship cell, you see, that we have a file name problem, which manifests itself in the say instruction.

Edit: You need to use the --script-verbose option to get a proper error message.

Looks like music/sound support is broken. (I use OSS4 by the way, maybe ALSA will work fine)

OpenMW tries to play the music over and over again, resulting in spam and a freezing X-Server. I tried to save the console output but it eats 22MB in just 3 seconds.

EDIT: I used head to show the first 2000 lines:

http://digi-thal.de/~fallenwizard/spam2.txt

Looks like music/sound support is broken. (I use OSS4 by the way, maybe ALSA will work fine)

OpenMW tries to play the music over and over again, resulting in spam and a freezing X-Server. I tried to save the console output but it eats 22MB in just 3 seconds.

Wow, that's a respectable amount of output. Something is definitely going wrong here. I'm not getting any of the output you're getting (after the "Playing /path/file.mp3" line.)

This output has to be coming from one of the libraries, either OpenAL or Audiere, our first job has to be figuring out which one. It's definitely not coming from any OpenMW code.

The "layer3..." lines might suggest that something is going wrong with the mp3 decoding, which would point to Audiere.

First off, what versions do you have? I get:

Code: Select all

$ pkg-config --modversion openal
1.12.854
$ audiere-config --version
1.9.4

terra / # pkg-config --modversion openal
1.12.854
terra / # audiere-config --version
1.9.4

Ok then that's not it.

I've created a new sound_test executable (in apps/soundtest/). Try pulling the newest version (remember git submodule update!) and see what happens (it should build automatically along with openmw.) The test uses the same sound code as OpenMW, but split it more steps so we can see exactly where it breaks down.

Run with:

Code: Select all

sound_test path/to/soundfile.mp3

BTW, the current strategy is to load the entire file into memory and play the raw sound data. This is a stupid strategy for music files obviously, and we'll fix it later, but it should still WORK.

I don't see anything relevant:
http://digi-thal.de/~fallenwizard/st_output.txt

Hmm, well that's actually relevant - it means that Audiere is crashing just by opening the file. May I ask where your audiere lib came from? If it's a distro-installed version, could you try compiling it manually? The amount of output you're getting suggests that you're running some kind of debug version of the library.

I could also try hooking it up with ffmpeg as input instead, maybe that will work.

BTW is anyone else having problems and/or success?

I don't get a crash. I don't hear anything either. But that is probably because I use the Redemption data directory and we have stopped all the default music by replacing all the explore files with dummy mp3s, that are completely silent. Testing via say instruction doesn't work, because we still have the path problem.

Edit: replaced the explore file with one of our custom music files and it plays properly.

I used this Ebuild to install audiere. I don't think it's a good idea to install it manually since it needs a ton of patches just to get it compiled with recent GCC versions.

I could also try hooking it up with ffmpeg as input instead, maybe that will work.

ffmpeg is a better idea since Audiere is unmaintained.
(Happy coding, the ffmpeg API is horrible)

EDIT:
What about Portaudio?
Looks like the perfect sound system for OpenMW

ffmpeg is a better idea since Audiere is unmaintained.

Done! Please try it now (you'll need to submodule update again.)

I've set up CMakeLists.txt so that it can compile either with Audiere or FFMpeg, check the settings at the top of the file. (Someone with more cmake knowledge than me can probably make this a bit nicer.)

(Happy coding, the ffmpeg API is horrible)

That's exactly why I made Mangle, so I'd only have to write that kind of nasty stuff once Luckily I've already added ffmpeg to Mangle earlier, so this was quick to set up.

However ffmpeg does give me some stuttering that I didn't get with audiere, can anyone else confirm?

EDIT:
What about Portaudio?
Looks like the perfect sound system for OpenMW

Interesting project. But it looks like it's a pure output library, no file decoding unfortunately, so it doesn't really help us. OpenAL already does sound output well enough.

[ 71%] Building CXX object apps/openmw/CMakeFiles/openmw.dir/__/__/libs/mangle/sound/sources/ffmpeg_source.cpp.o
In file included from /usr/include/libavutil/avutil.h:81:0,
from /usr/include/libavcodec/avcodec.h:30,
from /home/fw/openmw/libs/mangle/sound/sources/ffmpeg_source.hpp:10,
from /home/fw/openmw/libs/mangle/sound/sources/ffmpeg_source.cpp:1:
/usr/include/libavutil/common.h: In function â??int32_t av_clipl_int32(int64_t)â??:
/usr/include/libavutil/common.h:154:47: error: â??UINT64_Câ?? was not declared in this scope
/home/fw/openmw/libs/mangle/sound/sources/ffmpeg_source.cpp: In member function â??virtual size_t Mangle::Sound::FFMpegSource::read(void*, size_t)â??:
/home/fw/openmw/libs/mangle/sound/sources/ffmpeg_source.cpp:141:17: warning: â??int avcodec_decode_audio2(AVCodecContext*, int16_t*, int*, const uint8_t*, int)â?? is deprecated (declared at /usr/include/libavcodec/avcodec.h:3390)
/home/fw/openmw/libs/mangle/sound/sources/ffmpeg_source.cpp:142:69: warning: â??int avcodec_decode_audio2(AVCodecContext*, int16_t*, int*, const uint8_t*, int)â?? is deprecated (declared at /usr/include/libavcodec/avcodec.h:3390)
make[2]: *** [apps/openmw/CMakeFiles/openmw.dir/__/__/libs/mangle/sound/sources/ffmpeg_source.cpp.o] Error 1
make[1]: *** [apps/openmw/CMakeFiles/openmw.dir/all] Error 2
make: *** [all] Error 2

ffmpeg 0.6, x86_64

Doesn't compile for me.

In file included from /home/marc/OpenMW/openmw/libs/mangle/sound/sources/ffmpeg_source.cpp:1:
/home/marc/OpenMW/openmw/libs/mangle/sound/sources/ffmpeg_source.hpp:10:32: error: libavcodec/avcodec.h: No such file or directory
/home/marc/OpenMW/openmw/libs/mangle/sound/sources/ffmpeg_source.hpp:11:34: error: libavformat/avformat.h: No such file or directory
In file included from /home/marc/OpenMW/openmw/libs/mangle/sound/sources/ffmpeg_source.cpp:1:
/home/marc/OpenMW/openmw/libs/mangle/sound/sources/ffmpeg_source.hpp:19: error: ISO C++ forbids declaration of â??AVFormatContextâ?? with no type
/home/marc/OpenMW/openmw/libs/mangle/sound/sources/ffmpeg_source.hpp:19: error: expected â??;â?? before â??*â?? token
/home/marc/OpenMW/openmw/libs/mangle/sound/sources/ffmpeg_source.hpp:20: error: ISO C++ forbids declaration of â??AVCodecContextâ?? with no type
/home/marc/OpenMW/openmw/libs/mangle/sound/sources/ffmpeg_source.hpp:20: error: expected â??;â?? before â??*â?? token

I think I have identified the package I am missing, but Synaptic insists on uninstalling some other packages to install this one (and I think I still need those others). I don't feel like getting into this without some more research about what is going on or preferably not at all.

fw : /home/fw :: equery belongs avcodec.h
* Searching for avcodec.h ...
media-video/ffmpeg-0.6 (/usr/include/libavcodec/avcodec.h)

Afair Ubuntu uses another include path.

I made the requested improvements to CMakeLists.txt regarding sound options. Ready to be pulled.

@Nico: Regarding your TODO comments in MWSound::SoundManager:

// TODO: We need to attach it to the reference somehow. A weak
// pointer is probably the best bet

Do we really need a pointer from the CellRef to the sound entry in the SoundManager? That would blow up the CellRef even further and that is not good since we are going to have many of them. On the other hand we won't have many sounds active at the same time. I am no expert in this area, but I think the hardware usually can't even handle huge numbers of parallel sounds.

I suggest to store the sound list in the sound manager and with each entry store a MWWorld::Ptr. The only situation, where we need to go from the CellRef to the sound entry is when a reference is moved or disabled or deleted. These cases can easily be handled by searching though the sound list each time (since it is small and there usually isn't much moving/deleting/disabling per frame happening either).

TODO: We do not support
tracking moving objects yet, once a sound is started it stays in
the same place until it finishes.

This obviously has to be fixed at some point for player/npc
footstep sounds and the like. However, updating all sounds each
frame is expensive, so there should be a special flag for sounds
that need to track their attached object.

We only need to move sounds, when something moves. We can redirect all movement through a common interface (maybe as part of MWWorld::Class or MWWorld::World), that takes care of this. Really no point in doing anything on the sound side of the implementation.

Thanks for the comments. I meant to ask you about this.

I agree that keeping the references internal to the sound manager is a good idea. Your analysis on the speed of things seems sound (no pun intended.) I need some help on a couple of points though, since I've lost a bit of oversight of the new mwworld layout:

- what part of the reference should we use for indexing the objects? In your sample code you used reference.getRefData().getHandle() - I assume that is unique for all objects, or is it possible for two objects in a cell to have the same handle?

- how/where do we look up the soundId to get the corresponding filename?

Also I'm wondering exactly where to add the code for automatically attaching sounds to lights at cell loading. As far as I can see, the same playSound3D function can be used for this but I'm not sure where to call it from. I can probably figure it out though.

- what part of the reference should we use for indexing the objects? In your sample code you used reference.getRefData().getHandle() - I assume that is unique for all objects, or is it possible for two objects in a cell to have the same handle?

This is the Ogre handle. And yes, it is unique. But I suggest you just use a MWWorld::Ptr instead as the key. It has all the required operators.

- how/where do we look up the soundId to get the corresponding filename?

I am not sure about that either. The esm format has two sound related record types. loadsoun might work.

You access the ESMStore via MWWorld. That was what I originally passed the InterpreterContext for. But you were right in removing it, because it was overkill. You can pass either an MWWorld::Environment reference to the SoundManager (at construction) or directory a MWWorld::World reference.

Also I'm wondering exactly where to add the code for automatically attaching sounds to lights at cell loading. As far as I can see, the same playSound3D function can be used for this but I'm not sure where to call it from. I can probably figure it out though.

That is a bit complicated. playSound3D should work. But for a place to add it in, we probably have to go through the new class interface. I suggest you leave it to me.

Doesn't compile for me.
I think I have identified the package I am missing, but Synaptic insists on uninstalling some other packages to install this one (and I think I still need those others). I don't feel like getting into this without some more research about what is going on or preferably not at all.

You have the headers installed right? FFmpeg is notoriously inconsistent in where it puts its headers on different system. But with CMake we should be able to get it working. Turns out I'd forgotten to set up the include dirs in the cmake file though. Does the latest commit help?

Not really.

CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
FFMPEG_avcodec_INCLUDE_DIR
used as include directory in directory /home/marc/OpenMW/openmw
used as include directory in directory /home/marc/OpenMW/openmw/extern/caelum
used as include directory in directory /home/marc/OpenMW/openmw/extern/mygui_3.0.1
used as include directory in directory /home/marc/OpenMW/openmw/extern/mygui_3.0.1/MyGUIEngine
used as include directory in directory /home/marc/OpenMW/openmw/extern/mygui_3.0.1/OgrePlatform
used as include directory in directory /home/marc/OpenMW/openmw/apps/openmw
used as include directory in directory /home/marc/OpenMW/openmw/apps/esmtool
used as include directory in directory /home/marc/OpenMW/openmw/apps/soundtest
FFMPEG_avformat_INCLUDE_DIR
used as include directory in directory /home/marc/OpenMW/openmw
used as include directory in directory /home/marc/OpenMW/openmw/extern/caelum
used as include directory in directory /home/marc/OpenMW/openmw/extern/mygui_3.0.1
used as include directory in directory /home/marc/OpenMW/openmw/extern/mygui_3.0.1/MyGUIEngine
used as include directory in directory /home/marc/OpenMW/openmw/extern/mygui_3.0.1/OgrePlatform
used as include directory in directory /home/marc/OpenMW/openmw/apps/openmw
used as include directory in directory /home/marc/OpenMW/openmw/apps/esmtool
used as include directory in directory /home/marc/OpenMW/openmw/apps/soundtest

Ok :-/

Where are your ffmpeg headers installed?

That exactly is the problem. It seems they are not installed. And I can't install them. Synaptic refuses to. Unresolvable dependencies. No idea what is going on here. Maybe a problem with my system. I suggest I continue with Audiere and you can continue to work out the problem with FFMPEG together with fallenwizard. Since we have a proper sound system abstraction, it should not cause us additional work.

I used google a bit and the problem is: ffmpeg is not compatible with C++. The ffmpeg developers think C++ is not worth enough to use hacks/workarounds.

http://lists.mplayerhq.hu/pipermail/ffm ... 88093.html
I tried to use -D__STDC_CONSTANT_MACROS but it didn't work with OpenMW.
Any solutions?

*sigh* Time for plan C. (FFmpeg was plan B...)

I've added support for mpg123 now. If that doesn't work then I don't know what will. It might require a compete rebuild - at least it did for me, CMake was getting confused with all the options that kept changing. (And as always, a git submodule update is needed.)

This library only supports mp3 files, but it should work for the music at least. If this becomes our final choice then we can splice it with libsndfile later, which does everything EXCEPT mp3.

Let me know if it works!

/home/marc/OpenMW/openmw/libs/mangle/sound/sources/mpg123_source.cpp: In constructor â??Mangle::Sound::Mpg123Source::Mpg123Source(const std::string&)â??:
/home/marc/OpenMW/openmw/libs/mangle/sound/sources/mpg123_source.cpp:98: error: invalid conversion from â??const char*â?? to â??char*â??
/home/marc/OpenMW/openmw/libs/mangle/sound/sources/mpg123_source.cpp:98: error: initialising argument 2 of â??int mpg123_open(mpg123_handle*, char*)â??

*sigh* Time for plan C. (FFmpeg was plan B...)

I've added support for mpg123 now. If that doesn't work then I don't know what will. It might require a compete rebuild - at least it did for me, CMake was getting confused with all the options that kept changing. (And as always, a git submodule update is needed.)

This library only supports mp3 files, but it should work for the music at least. If this becomes our final choice then we can splice it with libsndfile later, which does everything EXCEPT mp3.

Let me know if it works!

OMG IT WORKS

Openmw compiled fine and I hear the music without stutter and other problems. mpg123 was the right solution

Good work and thanks!

OMG IT WORKS

Openmw compiled fine and I hear the music without stutter and other problems. mpg123 was the right solution

Good work and thanks!

Nice to hear that it works

@Zini: Oops, should be fixed now.

Confirmed. It works.

Also I'm wondering exactly where to add the code for automatically attaching sounds to lights at cell loading. As far as I can see, the same playSound3D function can be used for this but I'm not sure where to call it from. I can probably figure it out though.

That is a bit complicated. playSound3D should work. But for a place to add it in, we probably have to go through the new class interface. I suggest you leave it to me.

Actually it is more complicated than it is supposed to be. It seems there is still room for abstracting some more esm ugliness away. Will work on it in a separate branch. But I need a slight SoundManager interface change. It has been already pushed to my fork and you better pull it first, before you continue to integrate the sound manager.

Done. I think. Was quicker than I expected.

Something broke after the latest commit:

Data directory: /home/fw/openmw/data
Adding /home/fw/openmw/data/Tribunal.bsa
Adding /home/fw/openmw/data/Morrowind.bsa
Adding /home/fw/openmw/data/Bloodmoon.bsa
Loading ESM /home/fw/openmw/data/Morrowind.esm
loading cell 'Beshara'
[parse.c:529] error: Giving up searching valid MPEG header after (over) 64K of junk.

ERROR: Mangle::Mpg123 exception: No error... (code 0)

I was able to start OpenMW with: ./openmw --start "Balmora, Guild of Mages"
After I used "coc Addamasartus" it looked like this:

http://digi-thal.de/~fallenwizard/openmw.jpg

The coc problem could be my fault. I changed the way the rendering code is called and I must admit, I didn't test the coc instruction. Will investigate ...

I get the other error too. Sound related exceptions in several cells.

Something broke after the latest commit:

Data directory: /home/fw/openmw/data
Adding /home/fw/openmw/data/Tribunal.bsa
Adding /home/fw/openmw/data/Morrowind.bsa
Adding /home/fw/openmw/data/Bloodmoon.bsa
Loading ESM /home/fw/openmw/data/Morrowind.esm
loading cell 'Beshara'
[parse.c:529] error: Giving up searching valid MPEG header after (over) 64K of junk.

ERROR: Mangle::Mpg123 exception: No error... (code 0)

The latest commit only works with Audiere or ffmpeg unfortunately, since mpg123 can't read wav files. Sorry 'bout that. I'll hack in a wav reader in one of the next commits, either libsndfile or a custom one.

I don't know what's going on with the other bug...

EDIT: I added a try/catch block around it. Wav sounds won't work but cells should load now.

There is no other bug. Your workaround made it go away too. I guess it were the lights, which tried to insert a sound and caused an exception, which interrupted inserting references into the cell.

I've added in wav reader support (through libsndfile, which also handles ogg, flac and lots of other formats.) It should work for everybody now.

I've also set the Mpg123/libsndfile combo to be the default in CMakeLists, since this appears to be the most portable / available solution. I'm still allowing Audiere and FFMPEG as alternatives - it's only a few lines, since all the gory details are in the separate Mangle project.

.wav support works beautifully

Nice job!

Confirmed. Works here too. Looks like we have most of the sound implementation sorted out (at least the difficult part). Maybe we should start to look for volunteers to do the packaging for the upcoming 0.08 release.

i get segfaults after starting the game (nearly every time).
Here's the link to the created gist issue with the backtrace:
http://github.com/korslund/openmw/issues#issue/6

Strange. I don't have any problems of this kind. According to your backtrace the problem is deep inside mangle. I guess Nico will have to have a look at it. Not good! Not good at all! You did the required sub-module update, right?

Yes, and if it's not crashing, sound plays well.

EDIT: I'm using the ubuntu dev libs: libsndfile-dev_1.0.21-2, libmpg123-dev_1.12.1-0ubuntu1

I can confirm, I get crashes in exactly the same place. Seems to be a memory corruption bug, possibly a bug in one of the sound loaders. I'll look into it.

Ok, now I've fixed music streaming (notice how the music starts instantly now), and fixed a couple of other minor non-crash bugs that I could find. And it still crashes.

The part where it crashed earlier (BufferStream while loading music) is not even run anymore - which means this crash has moved on us. Looks like memory corruption somewhere. And now it's just a segfault without the trace.

However if I disable the non-music sounds, the crashes seem to go away (but it's hard to tell, I only get them once in a while anyway.)

I will keep looking into it.

EDIT: seems to be no crashes when using Audiere or FFmpeg.

Looks like the crash is in the ALSA part of libsndfile or mpg123. I am an OSS user and openmw runs without segfaults.

Looks like the crash is in the ALSA part of libsndfile or mpg123. I am an OSS user and openmw runs without segfaults.

I don't know, it seems very unlikely to me that it's related to the sound output (ALSO, OSS, etc) part of mpg123, since we're not using that part at all. Also libsndfile doesn't have any output component, it's a pure file reading and number crunching library. But it could be that either library is doing something strange that it shouldn't.

However if I disable the non-music sounds, the crashes seem to go away (but it's hard to tell, I only get them once in a while anyway.)

...

EDIT: seems to be no crashes when using Audiere or FFmpeg.

If I remember correctly most of the music sounds are mp3, while most of the non-music sounds are wav. If those two observations of yours are correct, this would indicate a problem related to libsndfile.

If I remember correctly most of the music sounds are mp3, while most of the non-music sounds are wav.

That's true. Music = mp3, sounds = wav.

I agree. Also, the first crash for me happened after adding libsndfile to the mix. My next step is to try a custom wav decoder instead and see if that helps.

Ok. tested custom wav loader, still crashes. On the plus side, this puts libsndfile in the clear, since I completely disabled it (even linking) for the test.

Since all the actual crashes so far have occurred at or aroud music loading (for me at least), the next suspect would be mpg123. Now I'm NOT going to write my own mp3 decoder but I'll look into alternatives. In the mean time I have disabled music when using mpg123.

Ah, found something:
http://sourceforge.net/tracker/index.ph ... tid=733194
and
http://freshmeat.net/projects/mpg123/releases/318444

Seems like there was a severe memory corruption bug in libmpg123, fixed in version 1.12.2. According to synaptic I have 1.12.1 ... anyone else with crashes (or who didn't get crashes) care to check their versions?

> mpg123 --version
mpg123 1.12.3

marc:~/OpenMW/openmw$ mpg123 --version
mpg123 1.4.3

No crashes at all. But the post you linked indicates, that this is a 32 bit bug. I am on a 64 bit system.

Oh, wait. There is a comment that says it did not show up before 1.10. So my mpg123 version might be too old to have it.

This looks very plausible then. Since this is then a library issue and no longer an issue with the OpenMW source, I'm re-enabling music and closing the issue in the issue tracker.

Too bad 1.12.1 is the version in the Ubuntu repos. I will try reporting it with the Ubuntu folks.

Regarding the "Finished sound system" commit: SoundManager::say is still broken (the path issue) and you have a stray mp3 playing at startup (mx_explore_5.mp3; probably from early testing), that you might want to remove before declaring the implementation finished.

One more check mark in the Roadmap! The sound system is finished. The manager keeps a list indexed by the objects that the sounds are attached to (through MWWorld::Ptr) and a string id. You can use this (ptr,id) pair to start sounds, stop sounds, look them up to see if they're playing etc.

The interface was designed by Zini and is well integrated into the scripting system. And that means that script-generated sounds now work - check out the sound of water drops in the default starting cell in the latest commit. All built upon a pretty solid sound manager in libs/openengine/sound that generates sounds on demand, auto-deletes them when no longer needed, and so on.

We still need a music player - now it justs starts mx_explore_5.mp3 at startup and plays it until its done.

Hope you didn't miss my previous post.

Regarding the music player: I think that it will be pretty simple for now. All we have to do is:

1. Play sounds from the explore folder in random order
2. When SoundManager::streamMusic is called, stop playing the current track and play the new track instead
3. After finishing this track, continue with 1.

Given that I've understood the path issue correctly (simply prepending datadir/Sound/ and converting slashes should be enough right?) then it should be fixed now.

Does not work here:

Error loading /home/marc/OpenMW/build/data/Sound/Vo/Misc/CharGenName1.wav, skipping.

Oh, I see! It's the damn case problem again. The file is chargenname1.mp3, so this fails on Linux.

Hmm. A solution I've used for this problem before is to search the dirtree at startup and store all the file names in a case-insensitive map. That way you can look up the filename in the map and it returns the full (real) path. The only disadvantage is that it doesn't capture run-time changes to the dir tree, but that's hardly a big problem. I think I'll start working on this now.

Try it now.

Nope. Still not working:

Sound file Vo\Misc\CharGenName1.wav not found, skipping.

Ah now I see it, it's not just case but also file ending. Just like they converted all their tgas to dds, they've converted many of the wavs to mp3. I'll add in some extra logic.

Works now. But we have another problem. The sound effects are so loud, that I can barely hear the NPCs speak. A good place to try this out is the starting cell (Imperial Prisonship or something).

Another problem: Sounds started by a script are not removed on cell change.

and that is also true for sound effects attached to lights. Looks like the whole removing sound on cell change business doesn't work.

Was only a minor iterator problem. It is fixed in my fork now.

I made the sound volume bug into an issue, so I don't forget about it:
http://github.com/korslund/openmw/issues/issue/8

I am guessing this is simply a matter of understanding what the minRange/maxRange values in the ESM::Sound struct are supposed to mean, and how to translate them into OpenAL settings.

More specifically: we are using the inverse distance model (OpenAL default), and currently minRange (multiplied with some factor) is set as AL_REFERENCE_DISTANCE, while maxRange (and some other factor) is set as AL_MAX_DISTANCE. However this is probably completely wrong, since the first of these (the reference distance) determines the fall-off factor per distance, and should probably be linked to maxRange not minRange. More experimentation is needed here, we could also try other OpenAL distance models.

More than you would ever want to know about OpenAL can be found here:
http://connect.creativelabs.com/openal/ ... cation.pdf

More specifically: we are using the inverse distance model (OpenAL default), and currently minRange (multiplied with some factor) is set as AL_REFERENCE_DISTANCE, while maxRange (and some other factor) is set as AL_MAX_DISTANCE. However this is probably completely wrong, since the first of these (the reference distance) determines the fall-off factor per distance, and should probably be linked to maxRange not minRange. More experimentation is needed here, we could also try other OpenAL distance models.

FWIW, Morrowind uses DirectSound3D (when audio is set to Accelerated), which only has the inverse distance model, and the amount of rolloff-per-unit can be changed using a global factor. In OpenAL, that factor is applied to individual sound sources. From my experience with Oblivion modding, I know a sound has a base decibel level (which would need to be converted before passed to OpenAL).. I don't know what Morrowind can specify.

I also have a custom DLL for Wine that implements DirectSound(3D) as an "accelerated" device using OpenAL, so I can see what kind of values vanilla Morrowind tries to set (though only the global properties, it would be extremely difficult to pick out a specific sound from them all).

Looking a bit more, it seems Morrowind sets the distance factor 0.05, and doppler factor to 2.0 in exteriors (1.0 in interiors). In OpenAL, these calls would be, respectively:

Code: Select all

alSpeedOfSound(343.3f / 0.05f); // distance-factor x == speed-of-sound 343.3/x
alDopplerFactor(2.0f); /*or*/ alDopplerFactor(1.0f);

Those only affect the doppler effect, though (the pitch-shifting that occurs from velocity). The rolloff factor seems to be left at 1.0.

The thing I found to note is that sounds seem to have a min-distance property in the 100~200 range, with a max-distance property in the 1000~4000 range (min=100, max=2000, seemed to be popular in a quick 5 second test). No idea how these map to the values stored in the data files, though.

These findings are very interesting Chris. Just to clarify, when you say min=100-200 and max=1000-4000, what OpenAL calls are you referring to?

The sound data in ESM data consists exactly of a volume, a min-range and a max-range (in addition to the sound file name.) If we're lucky they are a direct match to the values you're talking about above.

EDIT: The ESM values are byte values (0-255) so obviously there is some scaling here, but that might be all.

These findings are very interesting Chris. Just to clarify, when you say min=100-200 and max=1000-4000, what OpenAL calls are you referring to?

The AL_REFERENCE_DISTANCE and AL_MAX_DISTANCE.

The sound data in ESM data consists exactly of a volume, a min-range and a max-range (in addition to the sound file name.) If we're lucky they are a direct match to the values you're talking about above.

EDIT: The ESM values are byte values (0-255) so obviously there is some scaling here, but that might be all.

Hmm, I had a look in the Morrowind Construction Set, and it seems the min/max values are input directly (ranging from 0 to 255), and the volume is given as a floating point (so I guess it expands 0..1 to 0..255).

For the volume, I'm not sure if it's supposed to be treated as a linear gain (in which case just divide by 255.0 and give to OpenAL), or if it's a logarithmic scale, where 0 = -100dB (virtual silence) and 255 = 0dB (normal volume). In the case of the latter, it would need to be converted:

Code: Select all

gain = pow(10.0, (volume*100.0/255.0 - 100.0) / 20.0);

Or for all I know, 255 = -100dB and 0 = 0dB, in which case:

Code: Select all

gain = pow(10.0, (volume*-100.0/255.0) / 20.0);

The min/max values I'm clueless on. It seems there's some kind of modifiers as well as scaling going on, because a bunch of sounds have them at 0/0, even though they would attenuate in-game (the kagouti sounds, for example). The Lorkhan heart exploding sound has them set to 255/255, which effectively disables attenuation. Some of the other heart-related sounds have values like 40/255.

Unfortunately, it seems there isn't much in the way of information on Morrowind's CS, so I can't find out exactly what these values are supposed to do or how they translate in-game. I'll keep looking, but if anyone knows of any good tutorials/guides on Morrowind's CS (and includes a section on sounds), that would help.

One way to proceed would be to pick out 1-2 particular sounds in one cell, and trace how the DirectSound / OpenAL output from that compares with the data. (In the worst case it might be necessary to create a test case in the CS to isolate sounds completely.)

Okay, fooling around in the CS, I came to these conclusions:

• MinDist is multiplied by 20
• MaxDist is multiplied by 50
• If MinDist and MaxDist are 0, they default to 100 and 2000 respectively (ie, 5 and 40)
• If just MinDist is 0, it becomes 1 (no multiplying)
• If MaxDist < MinDist after multiplying, set MaxDist=MinDist

I haven't yet found a correlation with the volume. In fact, it just has me more confused as it doesn't seem to be linear or logarithmic. But I'll keep looking.

I think I got the volume now. I'm not sure how the CS value maps to the byte value in the esm/esp, but I found this by toying with the volume:

• 1.0 = 0 milliBels
• 0.75 = -837 milliBels
• 0.50 = -1674 milliBels
• 0.25 = -2510 milliBels

Essentially, this makes it a linear mapping of a logarithmic scale. 1.0 = 0mB, 0.0 = -3348mB. Only catch is that 0 seems to be special-cased to -10000mB (virtual silence).

So, if I had to guess, it would be along the lines of:

Code: Select all

if(volume == 0)
  gain = 0.0;
else
  gain = pow(10.0, (volume*3348.0/255.0 - 3348.0) / 2000.0);

Here's a patch that applies the changes I found above, and also fixes the toMp3 function to not assume the given file has an extension comprising of 3 letters.

I think there might be a bit more to the volume value, but it's probably good enough for now.

Thanks. I'll get to testing it soon.

Tested, works great!

I just tried openmw --start "Imperial Prision Ship", and the sounds are much more in balance now. If there are no objections I'll think I'll close issue 8 with this patch.