Neo Performance Guide
From NeoWiki
This is the Neo performance guide. It will attempt to guide you in how to design with and use Neo to achieve maximum performance, including configuration tweaks and OS stuff, etc.
Please note that this is a very early draft version. As of May 2008, it contains correct information, but it needs to be a bit more polished. If you find something you disagree with, please join in the discussions on the mailing list or feel free to edit it here directly.
Contents |
[edit] Neo primitives' lifecycle
Neo manages its primitives (nodes, relationships and properties) different depending on how you use Neo. For example if you never get a property from a certain node or relationship that node or relationship will not have the property loaded into memory. Another example is a node that you never request any relationships from, that node will then not load any of its relationships into memory.
Nodes and relationships are cached using LRU caches. If you (for some strange reason) only work with nodes the relationship cache will become smaller and smaller while the node cache is allowed to grow (if needed). Working with many relationships and few nodes results in bigger relationship cache and smaller node cache.
The Neo API specification does not say anything about order regarding relationships so invoking Node.getRelationships() may return relationships in different order than previous invocation, this allows us to make even heavier optimizations returning the relationships that are most commonly traversed.
All in all Neo has been designed to be very adaptive depending on how it is used. The (unachievable) overall goal is to be able to handle any incoming operation without having to go down and work with the file/disk I/O layer.
[edit] Basic memory management
If you increase heap size Neo will automatically cache more nodes,relationships and properties. By default Neo is set to use 77% of the heap, after that LRU caches will start to throw out elements for garbage collection. 77% heap usage with normal JVM settings on normal hardware achieves good overall performance avoiding "GC-trashing".
TODO: explain how to change the heap usage, possible to configure different settings for nodes and relationship caches.
If the application built on top of Neo makes use of all available heap memory you will get very poor performance since Neo will not cache anything and any work performed results in file/disk I/O.
[edit] Advanced memory managment
Neo makes heavy use of the NIO package that got introduced in Java 1.4. Native I/O may result in memory being allocated outside the normal Java heap. Also a well configured OS with large disk caches will help a lot. To get good performance on an enterprise OS such as Linux or Solaris you can start by investigating the size of the Neo database. Make sure you have performed a clean shutdown of the database before you sample the store directory (else logical log size will be included).
As an example we will use a fairly small but still substantial system (fairly typical of any small-to-medium-sized webapp) we've had running on Neo for the last 5 years. Currently the Neo store of that system is about 2GB in size. Lets have a look on some of the interesting files in the store directory:
14M neostore.nodestore.db 510M neostore.propertystore.db 1.2G neostore.propertystore.db.strings 304M neostore.relationshipstore.db
The nodestore stores information about nodes (not their properties or relationships), propertystore stores information of properties and all simple properties such as primitive types (both for relationships and nodes). The propertystore strings stores all string properties (that this particular system makes heavy use of) and finally relationshipstore holds all the relationships.
On the production machine (SunFire v20z, Solaris 10, Java 1.6) we have 4GB of memory. We've reserved about 2GB for the OS to be able to keep proper file system caches. The Java heap is set to 1.5GB, that leaves about 500MB for memory allocated outside the normal Java heap. In this particular system traversal speed is very important so we need fast access to nodes and relationships. The system has about 2M nodes, 10M relationships and 25M properties and the total size of the node and relationship store files are 318MB. Since traversal speed is very important we tell Neo to use memory mapped buffers for all of the node and relationship store and then divide the rest on the property and string stores.
TODO: explain howto set memory mapping.
[edit] JVM tweaking
TODO: talk about JVM tweaking, Neo likes -server.
[edit] Transactions
Too small = poor performance and lots of I/O wait, too big = poor performance because of memory usage. In the middle commits for large batch jobs.
[edit] Performance numbers
TODO: present some performance numbers from dozer
[edit] Second-level caching
As mentioned in the Guidelines for Building a Neo App, "always assume the node space is in memory", but sometimes it is necessary to optimize certain performance critical sections. Neo adds a small overhead even if the node, relationship or property in question is cached when you compare to in memory data structures. If this becomes an issue use a profiler to find these hot spots and then add your own second-level caching. We believe second-level caching should be avoided to greatest extend possible since it will force you to take care of invalidation which sometimes can be hard. But when all else fails you have to use it so here is an example of how it can be done.
We have some POJO that wrapps a node holding its state. In this particular POJO we've overridden the equals implementation.
public boolean equals( Object obj )
{
return underlyingNode.getProperty( "some_property" ).equals( obj );
}
public int hashCode()
{
return underlyingNode.getProperty( "some_property" ).hashCode();
}
This works fine in most scenarios but in this particular scenario many instances of that POJO is being worked with in nested loops adding/removing/getting/finding to collection classes. Profiling the applications will show that the equals implementation is being called many times and can be viewed as a hot spot. Adding second-level caching for the equals override will in this particular scenario increase performance.
private Object cachedProperty = null;
public boolean equals( Object obj )
{
if ( cachedProperty == null )
{
cachedProperty = underlyingNode.getProperty( "some_property" );
}
return cachedProperty.equals( obj );
}
public int hashCode()
{
if ( cachedPropety == null )
{
cachedProperty = underlyingNode.getProperty( "some_property" );
}
return cachedProperty.hashCode();
}
Problem now is that we need to invalidate the cached property whenever the some_property is changed (may not be a problem in this scenario since the state picked for equals and hash code computation often won't change).
To sum up, avoid second-level caching if possible and only add it when you really need it.
[edit] Good disks
As always, as with any persistence solution, performance is very much depending on the persistence media used. Better disks equals better performance, especially with non read-only transactions. In Neo a non read-only transaction will result in one flush during the commit phase (transaction log) and at some time later a second flush (grouped with other transactions) to the actual files storing the node space.
Neo supports JTA and two-phase commits so it is possible to enlist other XA enabled resources in the transaction but beware of this since adding more resources results in even more flushes (if non read-only) for the transaction.
TODO: add some performance numbers for minimal read-only transactions and minimal write transactions.
If you have multiple disks or persistence media available it may be a good idea to split the store across those disks. Moving the transaction log (and logical logs) to a disk that has a very low flush time is a good idea while the "big" store files make better use of a disk with low seek time.
TODO: explain howto configure Neo with different disks for tx log, logical log and store files.
The coming year will be an interesting one since the new flash storage disks with SATA interface are on the way. Hopefully these disks will not cost as much as the already existing solutions with hardware ramdisks that are quite expensive.
