The Week in Review 2012-07-30 - sir_herrbatka

Though there won’t be a new release for July, last week was a great one for OpenMW. So let’s recap.

First off, we have a new developer, _greye, who finished his first task (making CreatureStats into a class) in a single day. Then he picked another one (maintenance in the item dropping code) and finished that in a short period of time. Both were very well done, so thank you _greye and welcome to the team!

nhydock, another new developer, has taken the camera control task (which is composed of four or five subtasks).

scrawl’s new shader system has been merged into the main branch. What does that mean for OpenMW? Until now we have been using the Nvidia CG toolkit. This toolkit works with DirectX and OpenGL on all graphics cards, but only gives good performance with Nvidia cards (which is understandable since its developed by Nvidia). We need both DirectX and OpenGL support because Radeon graphics cards (built by amd) lack quality OpenGL drivers. Otherwise we’d just write our shaders in OpenGL. Again the problem is CG doesn’t provide good performance for the Radeon cards. To make matters worse, CG is closed source and available for on Linux, windows, OSX and not on BSD.

To avoid the above mentioned problems with CG, many game developers instead write all their shaders twice: in GLSL for OpenGL and in HLSL for DX or just ignore OpenGL and focus on DirectX.

But the situation is improving because Scrawl made a library called “shiny”. From now on we can write a shader once and then compile it as either GLSL or HLSL. But that’s not all, here’s scrawl detailing the new library’s design and features:

– High-level layer on top of OGRE’s material system. It allows you to generate multiple techniques for all your materials from a set of high-level per-material properties.

– Several available Macros in shader source files. Just a few examples of the possibilities: binding OGRE auto constants, binding uniforms to material properties, foreach loops (repeat shader source a given number of times), retrieving per-material properties in an #if condition, automatic packing for vertex to fragment passthroughs. These macros allow you to generate even very complex shaders (for example the Ogre::Terrain shader) without assembling them in C++ code.

– Integrated preprocessor (no, I didn’t reinvent the wheel, I used boost::wave which turned out to be an excellent choice) that allows me to blend out macros that shouldn’t be in use because e.g. the shader permutation doesn’t need this specific feature.

– User settings integration. They can be set by a C++ interface and retrieved through a macro in shader files.

– Automatic handling of shader permutations, i.e. shaders are shared between materials in a smart way.

– An optional “meta-language” (well, actually it’s just a small header with some conditional defines) that you may use to compile the same shader source for different target languages. If you don’t like it, you can still code in GLSL / CG etc separately. You can also switch between the languages at runtime.

– On-demand material and shader creation. It uses Ogre’s material listener to compile the shaders as soon as they are needed for rendering, and not earlier.

– Shader changes are fully dynamic and real-time. Changing a user setting will recompile all shaders affected by this setting when they are next needed.

– Serialization system that extends Ogre’s material script system, it uses Ogre’s script parser, but also adds some additional properties that are not available in Ogre’s material system.

– A concept called “Configuration” allowing you to create a different set of your shaders, doing the same thing except for some minor differences: the properties that are overridden by the active configuration. Possible uses for this are using simpler shaders (no shadows, no fog etc) when rendering for example realtime reflections or a minimap. You can easily switch between configurations by changing the active Ogre material scheme (for example on a viewport level).

– Fixed function support. You can globally enable or disable shaders at any time, and for texture units you can specify if they’re only needed for the shader-based path (e.g. normal maps) or if they should also be created in the fixed function path.

This library should make the lives of developers easier and since it is designed for the Ogre3d rendering engine, it could have bright future beyond even the OpenMW project. Huzzah for open source!

Take a look at the new water shader scrawl made using the “shiny” shader library.

 

 

If all that is not awesome enough, then wait dear readers, because there’s more.

In every blog post since time immemorial we have talked about getting animations to work, but now that saga’s beginning its conclusion. That’s right, the new and completed animation system awaits testing in the main branch. jhooks1 and Chris’s huge effort is bearing fruit. You can expect only falling action (bug fixes and performance improvements) from here on out.

Although animations have been present in OpenMW for some time, they did not use ogre’s animations system, instead OpenMW manually moved all bones. This was not efficient and not all creatures were correctly animated, especially those from expansions sets. For example, Bloodmoon’s werewolf animations looked really bizarre.

But now those problems are solved and we employ Ogre for the animations. Even hardware skinning can be added as a post 1.0 feature.

Zini is finishing up the potion usage task. Now the drinking sound effect plays correctly. Additionally,  zini refactored the action class.

Corristo made sure that OpenMW still works for you OSX users out there. There were mac specific bugs that should now be resolved.

We migrated to new servers after the result of the slashdot effect exposed our inability to handle heavy users loads (our servers crashed). The first choice was amazon cloud which looked quite nice, but soon it became clear that they are insufficient for our needs. At the moment OpenMW.org is on one MyDevil.net server, while bugtracker and forum are on OVH server. In addition OpenMW.org is protected by Cloudflare CDN “proxy”, so it has chance to handle next slashdot effect.

Fork me on GitHub
Week in Review 2012-07-22 - sir_herrbatka

Another week, another post for you, dear readers. This summary includes the most significant changes by our veteran developers, but there are some folks doing important work that is not credited here. So thank you to all our contributors.

Last week, I wrote about the shaders used in terrain rendering. This week scrawl, using shaders, reduced the amount of code for terrain implementation by 75%. THREE CHEERS FOR EFFICIENCY!!!

OpenMW employs shaders for our skybox, water, and soft shadows. Scrawl has worked on refining the first two, resulting in marked visual improvements over the original Morrowind. The process of refining shaders will continue, so expect many perty pictures and videos. Great work Scrawl!

Speaking of great work, Chris pushed a large number of code revisions (50!) towards integrating .nif animations into Ogre3d’s animation system. Once complete, this will prove to be a more efficient method for animations and boost OpenMW’s frames per second. It works… to some extent. Currently, animations are lacking integration with the physics system, some meshes (for unknown reasons) are not animated, and other meshes occasionally ( and seemingly randomly) are not animated. However, clothes (including robes) are working, as well as the majority of creatures. We cannot give you an exact date when it will be finished, but what needs to be done to get physics integration working is mostly understood. The other issues are cryptic and it is difficult to tease out their causes.

Meanwhile Zini, our fearless leader, finished the magical effects drain and fortify. Also, the bug in the auto-equip logic was fixed, as well as a few other bugs in various areas. Zini also refactored the code for dynamic statistics and merged a lot of branches into the main development branch.

Reverse engineering formulas is going well.  The ingredient eating formula appears close to completion (some things still need to checked and verified) thanks to Lazaroth. The acrobatics task is also progressing nicely, BrotherBrick is using his powerful server to calculate data points and Epsilon is a very dedicated worker, designing formulas to test. It may be finished as early as next week.

Week in Review 2012-07-16 - sir_herrbatka

Curse you, oh glorious sun! Vacations are not conducive to developing Free Software…

The programmers are still working, but the rapidity of new features being added has dropped and we have nothing shiny to show off this week. Progress is being made in technical aspects of the project and it’s good to explain these things from time to time.

OpenMW is a far more modern engine than Morrowind and Scrawl continues working on his shader system. Our engine uses shaders in rendering terrain, which should prove a more efficient method then the old Morrowind’s.

Chris is improving the handling of the nif file format. This is a prerequisite before we can finally animate meshes, and anything else related to nifnodes, correctly.

Zini is working on potions usage now that the possibility of creating potions has been recently added. It’s a small fragment of Morrowind gameplay, but this “step by step” method is what will allow us to finish the gigantic task that is OpenMW 1.0.

Ingredients usage was planned, but we still don’t have the right formulas, so that needs to wait a moment. No worries, the researchers are working on the formula as we speak.

The Week in Review 2012-07-09 - sir_herrbatka

As you probably noticed, the site was down. This was due to a sudden and sharp increase in page views that occurred when OpenMW made the news aggregating website slashdot.org. A veritable horde of readers clicked on the story, passed to our site, and soon all I could do as an admin was log into the server and watch with shock and awe as the heart of apache processes was torn out. Normally we get 10,000 page views a month, and because of slashdot we had 25,000 in one day.

 

However, we benefited as many new developers and contributors have discovered us and are now joining. So we’d like to say, Thank You Slashdot! And despite the setback of the site being down, the development team continued their work as usual.

Zini was working on the refactoring task when he found many unnecessary “includes” in the source code. Don’t worry readers about what includes are, all you need to know is that many were unnecessary and that translated to slow downs in compile times. Many users complain that it takes forever to compile and build OpenMW and while users could perhaps view compiling OpenMW as exercise in patience, they won’t have to as removing these “includes” improved build times by nearly half.

He also worked on proper skill development implementation. As of now, the only skill that can be seen in game is Alchemy (and even it’s not finished), but that will soon change.

Scrawl has been tinkering with shaders. Interestingly, he found that using GLSL shaders caused build time improvements twenty-one times. Since the number and complexity of shaders OpenMW uses can be described as “modest”, the impact on the total compile time is negligible, but it’s still intriguing.

jhooks1 is still working with animations.

Figuring out Morrowind’s formulas has finally gained momentum. Myckel is bravely working on alchemy, and he has a couple new colleagues on the forums which include Epsilon. Epsilon is developing scripts to speed up the testing phase exponentially. It’s great work and we welcome them to the team.

The week in review 2012-06-11 - ap0

Hello everyone !

The major news for the week comes via Mark (creator of the Crystal Scrolls): he nearly finished multiple esp/esm support! There are numerous problems, but things are progressing: you’ll soon be able to travel to Solstheim!

garvek, our new developer who is also part of the dugeonhack project, started working on cmake fixes and making OpenMW compile with MSVC. The DungeonHack team is deciding whether to help OpenMW reach basic functionality and then base DungeonHack off our engine. Sounds good to me!

zini returned to working on the script implementation. Excellent!

jhooks1 is continuing work on the physics system. Nothing really spectacular and the ogre3d enabled new animations system still is not able to handle robes and some creatures. If someone is an expert in animations then he is welcome on our forums.

Gentoo overlay for OpenMW has reached alpha status and awaits testing. Big thanks (if it’s working correctly) to edmondo.

Lurkers of the irc channel are reporting that scrawl said:

btw I have started with rewriting shader system to allow GLSL. I am a making sort of a library out of it, it will be similar to ogre RTShaderSystem but much better. then user can put in a settings file which shader language he wants 😛

Ho, and it seems like pvdk and k1ll are active again 🙂