During the past three months, the OpenMW team has been hard at work porting their codebase away from the Ogre3D engine and towards the OpenSceneGraph rendering toolkit. The previous post talks about our motivations.
We are pleased to announce the porting efforts are finally bearing fruit – as in, all gameplay-essential features have been ported across, so users can enjoy a legitimate game of Morrowind in the OpenMW-osg development branch.
Some advanced features – shaders, distant terrain, shadows, and water reflections – have not been ported yet. However, even this early, it is safe to say the transition has been a massive success, in more ways than we initially imagined. Players will find the new OpenMW loads faster, improves framerate, looks more like the original game, and fixes a host of long-standing bugs that proved difficult to address within our old framework.
Hold on… improved framerate? Let’s put that to the test…
The first benchmark
Our test environment shall be:
GeForce GTX 560 Ti/PCIe/SSE2, AMD Phenom(tm) II X4 955 Processor × 4, Linux 3.13.0-24-generic x86_64
1680×1050, full screen, no AA, 16x AF, no water reflections, no shadows, no shaders
Maximum view distance as per the in-game slider
The testing scene: good old (laggy) Balmora
Loading time avg.
System memory avg.
Unsurprisingly, the OSG port wins on all three counts. The framerate improvement is decent, though still far short of the 3-4x improvement we saw in earlier tests with a single model. There is no reason to be concerned, however – you should take these numbers with a grain of salt:
The comparison is unfair. New rendering features are included in the OSG branch, that bring us closer to vanilla Morrowind compatibility, but affect performance as well. For example, we now dynamically expand bounding boxes based on running animations, which fixes the infamous bug of cliff racers disappearing under certain angles (Bug 455), but is taxing on frame rate as well. Likewise, we have ditched static geometry batching, a performance optimization that was causing a multitude of issues, mainly incorrect lighting and scripted movement of objects not working. Even under all this new workload, the OSG port is still faster!
Expect higher performance gains for advanced graphical features. The comparison was made with the minimum graphics settings, for the simple reason that the advanced settings (shadows, reflections, etc.) do not exist in the OSG branch yet. We expect that once those advanced features are ported, their impact on framerate will be much lower than before, simply due to the new renderer scaling a lot better with regards to GPU workload. Draw submissions are now offloaded to a worker thread, so the graphical complexity of the scene does not block the main thread from performing work in the meantime such as culling, physics, scripting, and animation.
The real optimization phase hasn’t even begun yet. The main focus for now has been to get the game back to a playable state, and that only happened a few days ago. Now, there are plenty of optimization opportunities on the horizon. Our new rendering framework gives us more control over how the scene graph is structured, and how updates are dispatched, something we are only just starting to take advantage of. Planned optimizations in the near future include:
Move skinning updates to a worker thread.
Move particle updates to a worker thread.
Share state across different NIF files.
Enable culling for particle emitters/programs.
Integrate a model optimizer. Morrowind’s models unfortunately contain plenty of redundant nodes, redundant transforms, and redundant state, which impacts rendering performance. The original engine runs an “optimizer” pass over the models upon loading them into the engine. We should implement a similar optimizer pass. OpenSceneGraph provides the osgUtil::Optimizer that might prove useful for this very purpose.
Create a more balanced scene graph, e.g. a quad tree, to reduce the performance impact of culling and scene queries.
Rendering is not the only bottleneck. Assuming an N times faster renderer would lead to an N times faster OpenMW is wrong. We have other systems contending for frame time, and now that our renderer is faster, these other bottlenecks are becoming ever more apparent. In particular, the physics and animation systems are currently the worst two offenders. Some preliminary optimizations for these systems have made it into the OSG port, but we have no doubts there is more room for improvement.
With that said, better performance is certainly not the only change users can look forward to:
Non-uniform NPC scaling (Bug 814):
Certain NPCs are now scaled along their X and Y axes, giving them a more bulky appearance, just like in vanilla Morrowind. Previous versions of OpenMW were unable to support this kind of scaling, due to an Ogre3D limitation.
Improved rendering precision: solves large coordinate precision issues, exhibited by flickering/shaking when the player travels too far from the world origin. Video comparison
Removed static geometry batching: fixes incorrect lighting (Bug 385), fixes scripted movement of objects (Bug 602), and improves cell loading time.
Vanilla-accurate transparency settings: Previous versions of OpenMW were using transparency settings not quite accurate to vanilla Morrowind, in an effort to facilitate static geometry batching. That has been fixed – users will notice that foliage and other transparent objects now have softer edges.
Added small feature culling option: In addition to culling off-screen objects, we can now also cull objects that would be smaller than a single pixel when rendered. The visual change is virtually unnoticeable, so we have enabled the option by default.
NPC scaling comparison
Re-designed NIF loader
Support for non-uniform scaling in NIF files (Bug 2052)
Fixed limitation on the number of nodes in NIF files (Bug 2187)
Fixed animations “freezing” when the object is culled (Bug 2151)
Redesigned skinning algorithm to allow for a more efficient scene graph
Dynamically expand bounding boxes based on skeletal animation; fixes certain creatures disappearing under certain view angles (Bug 455)
NIF scene graphs are now a shared resource; drastically improves loading time.
Animation text keys are now a shared resource
When compiled with Bullet 2.83 or later, take advantage of btScaledBvhTriangleMeshShape for efficient shape instancing
Remove the “detailed” Bullet raycasting shapes, replaced by direct raycasting on the scene graph (see “New raycasting” section), vastly reducing memory usage
Use a btCollisionWorld in place of a btDynamicsWorld, to avoid unnecessary updates for functionality that we are not using
Map collision objects by pointer instead of by name
Use the osgUtil::IntersectionVisitor for direct raycasting on the scene graph
Support for raycasting against animated meshes, fixing inaccurate cursor selection for actors (Bug 827)
Rewrote item selection logic for the inventory preview doll to use raycasting instead of a selection buffer, improving response time
Improved loading screen
Loading screen now renders in the draw thread, thus no longer blocks the loading procedure
Increased loading screen FPS so the loading bar moves more smoothly
Upload OpenGL objects in a background thread while the area loads, using osgUtil::IncrementalCompileOperation
Improved SDL2 support
SDL2, the library we are using for cross-platform input and window creation, has been more closely integrated with the rendering system. Practical benefits for the user include:
The antialiasing option finally works on Linux systems (Bug 2014)
SDL2 is now responsible for creating the graphics context, meaning new display APIs, such as Wayland and Mir on Linux, are automatically supported with no extra maintenance overhead for the OpenMW team.
8x antialiasing in action
A nice side effect of using OpenSceneGraph is access to their top-notch profiling tools. With the ‘F3′ key, an on-screen overlay appears that shows more information with every key press.
Profiling overlay – the colored bars represent OpenSceneGraph’s internal threads, the white bars OpenMW logic
The new OpenMW sports a unified OpenGL renderer on all our platforms. Rendering via Direct3D is no longer supported, easing the maintenance and support overhead for the OpenMW team.
Practically speaking, we have thus “fixed” Bug 2186 (garbage pixels on the minimap on Windows) and Bug 1647 (crash when switching to windowed mode on Windows).
Finally, we do have some bonus changes that are not strictly related to the OpenSceneGraph transition, but were committed to the OSG branch anyway in an effort to reduce merge conflicts:
Tweaked the activation range for lights (Bug 1813)
Implemented the NiPlanarCollider particle processor (Bug 2149)
Added UI scaling option:
Normal UI scale
2x UI scale, same resolution
Code maintenance / restructuring / cleanup
The codebase has considerably lost weight, which is likely interesting to developers, though not so interesting to end-users.
Physics code moved to a new “mwphysics” subsystem
Removed scene node names, i.e. the RefData::getHandle identifier
Removed platform wrapper
In total, ~23.000 lines of code were removed:
git diff upstream/master --shortstat
689 files changed, 24051 insertions(+), 47695 deletions(-)
Phew, that was a lot to take in – even at this stage, the list of improvements is massive, so our first priority should be to merge the port into the main branch, get our various nightly builds up and running again, then get a new release out.
Speaking for the longer term, we are not even close to unleashing the full potential of our new rendering engine. The next steps would be to further improve performance, then work on restoring shaders, distant terrain, water reflections and shadows. Our new NIF loader facilitates the implementation of background cell loading, which was originally planned as a post-1.0 improvement – now, it would be trivial to do, so we might see this feature pre-1.0 after all.
In the meantime though, on the graphics front, there is now nothing stopping us from releasing the long awaited OpenMW 1.0, so perhaps efforts should be diverted to fix the remaining OpenMW 1.0 blockers first, get version 1.0 out the door, then port the advanced graphical features for version 1.1. We might decide this on a case-by-case basis. In particular, the water reflections should be a lot easier to port than shadows, which weren’t working particularly well in the Ogre branch anyway.
If you have an opinion on the matter, feel free to comment. Though regardless of the priorities we decide on, these are certainly exciting times ahead!
The OpenMW team announces the release of maintenance version 0.36.1. This version addresses a regression that caused additional startup scripts to fail to launch. Grab it from our Downloads Page for all operating systems.
Crash when trying to view cell in render view in OpenMW-CS on OSX
Crash when switching from full screen to windowed mode on D3D9
Fixed additional startup scripts failing to launch
The OpenMW team is proud to announce the release of version 0.36.0! Grab it from our Downloads Page for all operating systems. This release is relatively small as we eagerly await scrawl’s rendering port to OSG. There are a multitude of important fixes for OpenMW-CS, which is progressing at a healthy pace. See below for the full changelog.
Crash when trying to view cell in render view in OpenMW-CS on OSX
Crash when switching from full screen to windowed mode on D3D9
Implemented hotkey for hand-to-hand
Implemented key bind for always sneaking
Implemented multiselection of entries in the data files list of the launcher
Implemented using application support directory as the user data path on OS X
Fixed Erene Lleni not wandering correctly
Fixed installer not detecting default directory correctly in 64-bit Windows registry
Fixed issue where cloning the player would prevent player movement
Fixed casting bound weapon spell not switching to “ready weapon” mode
Fixed enchanted item charges not updating in the spell list window if it is pinned open
Fixed behavior when fatigue falls below zero due to the drain effect
Fixed error on startup with Uvirith’s Legacy enabled
Fixed sex, race, and hair sliders not initializing properly in chargen
Fixed minor issues with terrain clipping through statics
Fixed invisible sound mark having collision in Sandus Ancestral Tomb
Fixed tooltip display on stolen gold stack
Fixed persuasion mechanics using player stats instead of NPC’s
Fixed not defaulting to world origin when an unknown cell is encountered
Fixed NPCs not reacting to theft when player has the inventory open
Fixed water z-fighting issue
Fixed GetSpellEffects to better emulate vanilla behavior
Fixed a dialog choice in mod Rise of House Telvanni not exiting out of dialog
Fixed Detect Spell returning false positives
Fixed the map icons used by Detect spells
Fixed a crash on first launch after choosing “Run installation wizard”
OpenMW-CS: Implemented global search & replace
OpenMW-CS: Implemented dialog mode only columns
OpenMW-CS: Implemented optionally showing a line number column in the script editor
OpenMW-CS: Implemented optionally using monospace fonts in the script editor
OpenMW-CS: Implemented focusing on ID input field on clone/add
OpenMW-CS: Implemented handling moved instances
OpenMW-CS: Implemented a start script table
OpenMW-CS: Implemented a start script record verifier
OpenMW-CS: Fixed changed entries in the objects window not*? showing as changed
OpenMW-CS: Fixed changing certain entries not being saved in the omwaddon file
OpenMW-CS: Fixed terrain information not saving
OpenMW-CS: Fixed files with accented characters not being listed
OpenMW-CS: Fixed cloning/creating new container class resulting in an invalid omwaddon file
OpenMW-CS: Fixed cloning omitting some values
OpenMW-CS: Fixed crash when issuing undo command after the table subview is closed
OpenMW-CS: Fixed being unable to undo a record deletion in the object table
OpenMW-CS: Fixed multithreading for operations
OpenMW-CS: Changed References/Referenceables terminology to Instance/Object
As the feature list is getting shorter and shorter, our development team’s attention has been shifting towards issues that are a lot bigger than things like NPCs not doing what they are supposed to do.
Scrawl, for example, is slaving away at porting the project from Ogre3D to OpenSceneGraph, a relatively unknown but immensely flexible and expansive 3d engine which will not only allow OpenMW to run smoother than before, but also allows for better handling of materials and stitching the bodyparts of NPCs together (which is not nearly as gruesome as it sounds). Scrawl himself has told us about the what and why of OSG, and yes, I could tell you all that he is making good progress, but pictures speak louder than words.
It’s almost a throwback to version 0.13.0, but don’t worry; unlike the project way back when in 2012, much of the program is already in place right now, so we promise it won’t take three years to catch up. I mean, just between the time that I started writing this update and the time this got posted Scrawl got NPCs to render. Things are progressing fast.
Meanwhile, OpenMW-CS has been developing nicely as well. Our very own modding suite has gotten two more features that you have to see if you are a modder.
A plenty large code submission by cc9ii, based on some hard work by sirherrbatka, will give you access to stuff that you didn’t have access to before. Take NPCs, for example. Their settings were previously accessible only through the main table, but there are some things that were not contained therein. Selecting Edit record from the right click menu would bring up some more stuff to change (such as if this NPC would buy certain items), but cc9ii’s contribution expands that even further. By editing the NPC, you can now put stuff in his inventory, check out the paths they take while walking through the world, and much more.
Zini has announced with the proper fanfare that Global Search is now working. When confronted with a huge list of stuff, trying to find that lovely lime-ware platter from the Census and Excise Office is like searching for a needle in a haystack. Previously, it was possible to filter using a specific syntax, but why make it difficult? With Global Search implemented, you can instantly find that one object with nary a thought. What’s more, this global search function will be extended with a replace function, so you don’t have to edit all those potions one by one.
Finally, a bit of community stuff as well. Sterling of the Old School RPG YouTube channel has started his Let’s Play of Morrowind through the OpenMW engine. If you want to see how LPs are done or are just curious about how OpenMW runs in practice, go ahead and watch the first video below. Don’t forget to check out his channel afterwards for more content.
That it’s for this update. The team is ever active, and we can’t wait to tell you more in our next update. See you next time.
OpenMW is an attempt to reimplement the popular role playing game Morrowind. It aims to be a fully playable, open source implementation of the game. You must own Morrowind to use OpenMW. You can watch short video-faq or read detailed information on our FAQ page. OpenMW is an attempt to remake the Morrowind - brilliant role playing game. We're trying to remade Morrowind using open source tools and libraries. You won't need any Morrowind patches, because OpenMW will fix most common Morrowind engine bugs. It's worth to mention that OpenMW is open source Morrowind engine remake which allow much greater modability: change game rules, create new spell effects, etc. through scripting. With OpenMW you will be able to run Morrowind overhaul mods, texture replacers and much more like with original engine.