OpenMW 0.37.0 Released! 2015-11-30 - raevol

The OpenMW team is proud to announce the release of version 0.37.0! Grab it from our Downloads Page for all operating systems. This release brings the long-anticipated implementation of the OpenSceneGraph renderer! More info on this 3d graphics toolkit can be found on the OSG website. Hats off to scrawl for the Herculean amount of effort he put in working on this massive codebase change! The new renderer brings a massive performance speedup, as well as many graphical fixes and improvements.

You may notice that shadows are not re-implemented at this time, as well as distant land and object shaders, but we wanted to get the release out rather than delay any further! These features will be re-added soon! This release brings many other changes and bugfixes, as well as a huge amount of new work done on OpenCS, the editor for OpenMW. Some features are missing from OpenCS as well: only basic camera controls are implemented, pathgrid and cell marker rendering is missing, as well as instance moving. See below for the full changelog.

Changelog:

Want to leave a comment?
Fork me on GitHub 2015-11-30 - raevol - One pingback.
First of all… 2015-09-25 - Okulo

Hello again. Small update since it’s been a while since the last one. The move to OSG has called for a lot of work left and right. Most of it is not exciting enough to get into in a newspost (interesting enough if you want to delve deeper into the development of this project, though!), but there are a couple of things worth noting. First is the object selection via the scene view in OpenMW-CS. Scenes may contain a lot of stuff, and finding the exact item you need may be a hassle. Scrawl and Zini have been working on a way to use the scene window to select objects. Next up is selecting terrain and stuff that isn’t rendered in the scene (such as pathgrids and cell markers). You can’t move them around yet, though, that’s all for a different update.

If you’re not a modder, but only play the game, you will enjoy this second bit of news: Scrawl has restored sun flares! Check out the before and after pics:

Rather than being a flat disk in the sky, light blooms now. Pretties.

Every step in the direction of beautifying the game is a good one and the team (especially OSG/shader magician Scrawl) is planning to do much more! There’s the water shaders, and there are object shaders. The latter may be a bit more difficult than just using an already existing implementation, as performance might be negatively impacted too severely to just plop it in and go, so some work on that is required. After the object shaders are done, work on object shadows can start, something that doesn’t seem all that significant, but it’s one of those touches that makes a ton of difference once you see it. Definitely more on that in a later version.

Disposing of bugs is, of course, also a staple of Morrowind releases. Counting only the bugs on the tracker – so not including all the bugs that have been fixed as soon as they were encountered – 149 bugs have been fixed so far for this release. The other 40 issues on the tracker that have been closed are things such as small optimizations, a scaleable GUI, and several OpenMW-CS features.

Doing so has left OpenMW with less than 100 issues until 1.0. Of course this amount fluctuates wildly and some issues are relatively small while others are huge. Still, it’s great to see the finish line drawing nearer and nearer.

I’m not going to lie to you, though: porting is a toughie and OpenMW-CS has come to that point where nearly all basic functionalities are in place. That means that the next version won’t be released next week or even the week after that. But still, the team is working hard: they are committed to pushing out the next release with terrain optimizations and a basic water shader for your enjoyment.

Troubling as the time since our last release may seem, know that the team is still busy and that the work put in the next version brings one of the most significant changes for players since the implementation of terrain and physics. Thanks for your patience and see you next time.

Want to leave a comment?
2015-09-25 - Okulo - No pingbacks so far.
Fundamentals 2015-07-28 - Okulo

Those who have the ability to build OpenMW from the OpenSceneGraph branch have probably already tried it. The new OpenMW is streamlined, fast and of course, most importantly, very functional. It’s not very pretty yet, though. The Ogre version has a bunch of shaders, including Scrawl’s own gorgeous “Shiny” shader which makes the water look incredibly good, but they couldn’t be easily ported to the OSG version. But that’s just eyecandy, and though that is one step back, it’s only temporary. Add to that the fact that with its huge progress in terms of speed and the many fixes that it brings, the whole OSG port represents three steps forward. In fact, it’s far along enough to be included in the next release. So get ready, 0.37.0 will be the version that brings OpenMW-OSG to the world!

At the time of writing, 0.37.0 closes a whopping 138 issues on our tracker. As mentioned in the last post by Scrawl, NPCs are finally the right size. Raindrops won’t make fire see-through anymore and fire no longer make stuff look black. Your followers also won’t be squealing when you touch something the wrong way, so no more “friends” ratting you out to the popo just because you took something that is technically not entirely yours.

Newcomer Kunesj has created a fun little extra that has been integrated already: the crosshair and tooltips can now show if something is owned or not. It doesn’t work as well as it should yet, so for now it requires editing some configuration files to enable and configure this feature, but it’s a good start to something that a lot of players will welcome.

There’s been some work on OpenMW-CS. NPCs should now have a whole lot more stuff for you to edit, including faction rank, level, skills, and the like. UI nags such as the starting size of the main window and a cancel button closing OpenMW-CS rather than returning to the last screen have been resolved. You can now also sort your search results, which should make the global search function even more useful than before. And, oh, bonus! – we also have a pretty colour picker now!

So as you can see, the team is marching on, but besides the engine and the CS, two other projects have seen the light of day.

For the first one we should give a shout-out to TravDark, the creator of Ashes of the Apocalypse mod, and Envy123, the creator of an unofficial expansion to AoA called Desert Region 2. These two creators have generously allowed their work to be expanded and revamped so that they can be run as stand-alone games on the OpenMW engine. To this end, content in these mods needs to be replaced, and such an effort has already started. In order to be stand-alone, however, it requires replacement of all assets made by Bethesda and work from other modders. That’s a ton of work and a solution needs to be found.

Which brings us to the other project: a bare minimum package for game developers. As OpenMW is merely an engine, it doesn’t only run Morrowind. To have a working game on OpenMW, you need some bare minimum files, such as a sky and some animations. Enter the OpenMW Game Template project, a project which aims to provide a minimum package to run OpenMW. Aside from being a good foundation for game developers to work off of, it’s also a fantastic test run for OpenMW. When people try to create their own game, they could run into all sorts of issues and with the tight connections the Game Template project has to the development team, these basic issues can be documented and/or resolved before they can get in the way of the developers of full-fledged games.

So there it is. Two new projects on the side, the CS getting expanded, and OpenMW proper switching to a whole new graphical engine with major fixes and speed as a bonus. Plenty to be excited about, even in this late stage of development!

Want to leave a comment?
2015-07-28 - Okulo - No pingbacks so far.
OpenSceneGraph port now playable 2015-06-07 - scrawl

During the past three months, the OpenMW team has been hard at work porting their codebase away from the Ogre3D engine and towards the OpenSceneGraph rendering toolkit. The previous post talks about our motivations.

We are pleased to announce the porting efforts are finally bearing fruit – as in, all gameplay-essential features have been ported across, so users can enjoy a legitimate game of Morrowind in the OpenMW-osg development branch.

Some advanced features – shaders, distant terrain, shadows, and water reflections – have not been ported yet. However, even this early, it is safe to say the transition has been a massive success, in more ways than we initially imagined. Players will find the new OpenMW loads faster, improves framerate, looks more like the original game, and fixes a host of long-standing bugs that proved difficult to address within our old framework.

Hold on… improved framerate? Let’s put that to the test…

The first benchmark

Our test environment shall be:
GeForce GTX 560 Ti/PCIe/SSE2, AMD Phenom(tm) II X4 955 Processor × 4, Linux 3.13.0-24-generic x86_64
1680×1050, full screen, no AA, 16x AF, no water reflections, no shadows, no shaders
Maximum view distance as per the in-game slider

osg_bench
The testing scene: good old (laggy) Balmora

OpenMW OpenMW-osg
Framerate avg. 49 75
Loading time avg. 7s 3.4s
System memory avg. 344.6mb 277.1mb

Unsurprisingly, the OSG port wins on all three counts. The framerate improvement is decent, though still far short of the 3-4x improvement we saw in earlier tests with a single model. There is no reason to be concerned, however – you should take these numbers with a grain of salt:

  1. The comparison is unfair. New rendering features are included in the OSG branch, that bring us closer to vanilla Morrowind compatibility, but affect performance as well. For example, we now dynamically expand bounding boxes based on running animations, which fixes the infamous bug of cliff racers disappearing under certain angles (Bug 455), but is taxing on frame rate as well. Likewise, we have ditched static geometry batching, a performance optimization that was causing a multitude of issues, mainly incorrect lighting and scripted movement of objects not working. Even under all this new workload, the OSG port is still faster!
  2. Expect higher performance gains for advanced graphical features. The comparison was made with the minimum graphics settings, for the simple reason that the advanced settings (shadows, reflections, etc.) do not exist in the OSG branch yet. We expect that once those advanced features are ported, their impact on framerate will be much lower than before, simply due to the new renderer scaling a lot better with regards to GPU workload. Draw submissions are now offloaded to a worker thread, so the graphical complexity of the scene does not block the main thread from performing work in the meantime such as culling, physics, scripting, and animation.
  3. The real optimization phase hasn’t even begun yet. The main focus for now has been to get the game back to a playable state, and that only happened a few days ago. Now, there are plenty of optimization opportunities on the horizon. Our new rendering framework gives us more control over how the scene graph is structured, and how updates are dispatched, something we are only just starting to take advantage of. Planned optimizations in the near future include:
    • Move skinning updates to a worker thread.
    • Move particle updates to a worker thread.
    • Share state across different NIF files.
    • Enable culling for particle emitters/programs.
    • Integrate a model optimizer. Morrowind’s models unfortunately contain plenty of redundant nodes, redundant transforms, and redundant state, which impacts rendering performance. The original engine runs an “optimizer” pass over the models upon loading them into the engine. We should implement a similar optimizer pass. OpenSceneGraph provides the osgUtil::Optimizer that might prove useful for this very purpose.
    • Create a more balanced scene graph, e.g. a quad tree, to reduce the performance impact of culling and scene queries.
  4. Rendering is not the only bottleneck. Assuming an N times faster renderer would lead to an N times faster OpenMW is wrong. We have other systems contending for frame time, and now that our renderer is faster, these other bottlenecks are becoming ever more apparent. In particular, the physics and animation systems are currently the worst two offenders. Some preliminary optimizations for these systems have made it into the OSG port, but we have no doubts there is more room for improvement.

With that said, better performance is certainly not the only change users can look forward to:

Preliminary changelog

Rendering improvements

osg_scaling osg_transparency
NPC scaling comparison Transparency comparison

Re-designed NIF loader

Physics rewrite

New raycasting

Improved loading screen

Improved SDL2 support

SDL2, the library we are using for cross-platform input and window creation, has been more closely integrated with the rendering system. Practical benefits for the user include:

osg_antialiasing
8x antialiasing in action

Profiling overlay

A nice side effect of using OpenSceneGraph is access to their top-notch profiling tools. With the ‘F3’ key, an on-screen overlay appears that shows more information with every key press.

osg_profiling
Profiling overlay – the colored bars represent OpenSceneGraph’s internal threads, the white bars OpenMW logic

Passive fixes

The new OpenMW sports a unified OpenGL renderer on all our platforms. Rendering via Direct3D is no longer supported, easing the maintenance and support overhead for the OpenMW team.

Practically speaking, we have thus “fixed” Bug 2186 (garbage pixels on the minimap on Windows) and Bug 1647 (crash when switching to windowed mode on Windows).

Miscellaneous changes

Finally, we do have some bonus changes that are not strictly related to the OpenSceneGraph transition, but were committed to the OSG branch anyway in an effort to reduce merge conflicts:

osg_ui_scale_1 osg_ui_scale_2
Normal UI scale 2x UI scale, same resolution

Code maintenance / restructuring / cleanup

The codebase has considerably lost weight, which is likely interesting to developers, though not so interesting to end-users.

In total, ~23.000 lines of code were removed:

git diff upstream/master --shortstat
689 files changed, 24051 insertions(+), 47695 deletions(-)

What’s next?

Phew, that was a lot to take in – even at this stage, the list of improvements is massive, so our first priority should be to merge the port into the main branch, get our various nightly builds up and running again, then get a new release out.

Speaking for the longer term, we are not even close to unleashing the full potential of our new rendering engine. The next steps would be to further improve performance, then work on restoring shaders, distant terrain, water reflections and shadows. Our new NIF loader facilitates the implementation of background cell loading, which was originally planned as a post-1.0 improvement – now, it would be trivial to do, so we might see this feature pre-1.0 after all.

In the meantime though, on the graphics front, there is now nothing stopping us from releasing the long awaited OpenMW 1.0, so perhaps efforts should be diverted to fix the remaining OpenMW 1.0 blockers first, get version 1.0 out the door, then port the advanced graphical features for version 1.1. We might decide this on a case-by-case basis. In particular, the water reflections should be a lot easier to port than shadows, which weren’t working particularly well in the Ogre branch anyway.

If you have an opinion on the matter, feel free to comment. Though regardless of the priorities we decide on, these are certainly exciting times ahead!

Want to leave a comment?
2015-06-07 - scrawl - 2 pingbacks.
OpenMW Maintenance Version 0.36.1 Released! 2015-06-05 - raevol

The OpenMW team announces the release of maintenance version 0.36.1. This version addresses a regression that caused additional startup scripts to fail to launch. Grab it from our Downloads Page for all operating systems.

Known Issues:

Changelog:

Want to leave a comment?
2015-06-05 - raevol - No pingbacks so far.