psulek wrote:

Fine tests :-) This is the best product promotion when user(s) tests DO4 themselfs and found that DO4 is better than other similar products :-)

My opinion is that DataObjects.NET is the BEST ORM framework in the world ever, like DevExpress are the UI (at least web/win) in the world, as FastReports was the best reporting engine for Delphi :-)

Alex (Xtensive) wrote:

I've just read the article - some facts are surprising even for me (we made some tests as well), e.g. 6MB EF queries.

Most likely, all the differences related to inheritance here comes from fact we diffrently handle it:

• Most of ORM tools, including EF and NH, are designed to deal with relatively small inheritance hierarchies (may be 1..6 subclasses). Coult of joins in queries there is proportional to count of classes in hierarchy. That's generally quite bad, since query optimizer is simply unable to produce best join plan in this case: non-heuristical (= providing guaranteed result) algorithms for join order optimization require expotential time (t ~= exp(N)), so they're applicable only when the number of joins is small (let's say, <10). But if the number of joins is large, SQL Server (as well as any other DB) will use a heuristical algorithm to produce join plan. It provides a good result with high probability, although it isn't guaranteed such a result will be the best possible one. I remember that some heuristical algorithms require t ~= N^2 time, which is incomparable to expotential time on big N. So this is the case when SQL Server trades performance on query execution for performance on query plan generation - actually, it simply has no other choice here.
• DO is designed to deal with hierarchies of generally any hierarchis. The could of joins in queries for such hierarchies is proportional to height of a hierarchy, which is nearly constant (e.g. <10) even for large models.

Btw, is it possible to get source code of your test? We'd like to study the queries sent by EF here to fully understand the reasons & write more detailed blog post about this (we can do this by our own, but if there is something that already works, we'd prefer to avoid unnecessary wastes).

And, certainly, I'd like to share a link to your post in DO4 blog - is this possible? Actually, that's one more reason for me to study the source code. I want to be sure everything is fully honest here, and there are no any errors...

Alex (Xtensive) wrote:

Few more notes:

1) Some quotes are really, err... Astonishing:

We asked Microsoft for guidance and the simple but disturbing answer was: "Just because you can does not mean that you should” and “Don’t do that!”. The SQL produced by the EF was unreadable and huge (some queries mounted up to 6 Mb!!) Our preliminary tests show that NHibernate is about 250 times faster than the EF with Inheritance. For example: Loading all data needed for 1 Dossier took over 6 seconds with EF and 286 ms with NHibernate on the first load. The second load of a dossier still takes 280 ms in EF and under 1 ms with NHibernate.

2) You should know that you can omit .Prefetch in case with DO4, if it isn't really necessary (i.e. you don't all the inherited fields). Moreover, since .Prefetch is based on IEnumerable<t>, but not IQueryable<t>, you can build prefetch sequence containing only necessary entities by filtering the original sequence and relying on already fetched properties there (type, fields and so on).

3) Most likely, DO4 will be slower on relatively small inheritance hierarchies (1...6 types): currently we don't fallback to "EF way" (= join everything). Although we're planning to add this in future - this isn't done mainly because there were no such requests. So as summary:

• Most of ORMs behave nearly the same, if there is no inheritance. Performance they show in this case mainly depends on query compilation and materialization speed. If you're interested in details, see our tests at ORMeter.net.
• If you deal with big inheritance hierachies (in terms of count of classes there), or there is a possibility you'll have them in future, DO4 will be a better choice then e.g. EF and NH. DO4 is initially designed to behave well in this case; moreover, we know very well what's necessary here, since one large hierarchy was the only supported case in previous versions of DO; we know a set of installations successfuly running >10GB databases with >500 persistent types in such a configuration (table per class inheritance mapping).
• If you deal with small inheritance hierachies, currently EF might provide better results, since it fetches all the fields declared in the hierarchy on queries, but DO fetches just fields of type you query for (i.e. declared fields) and all the inherited fields (so possibly you'll need an additional .Prefetch in DO4 - this depends on your query). I suspect NH won't be able to compete anyway here: it materializes just about 40K tiny entities per second (see "LINQ Materalize" test there), which is noticeably less then even .Prefetch sequence processing in DO4 (.Prefetch relies on batches & set-based queries to fetch the data it needs). On the other hand, this gap will be anyway closed shortly. Currently I recommend you to either avoid using inheritance in this case (that's normally relatively simple, if the count of types in hierarchy is small), or simply rely on provided performance, and if you'll feel it isn't good enough, kick us here.

Alex (Xtensive) wrote:

This is the best product promotion when user(s) tests DO4 themselfs and found that DO4 is better Fully agree. Actually, ORM tests are pretty complex: there are lots of products, and each of them is quite individual. So people implement such tests themselves quite rarely - I suspect, many tried, but only some were able to come to an end; others simply stick to e.g. the first tool they considered good enough. I clearly imagine how much efforts it costs to test even few of products on some basic case, not speaking about real-life cases.

On the other hand, lots of people would really like to see some real-life test - we know this, because ORMeter (formerly ORMBattle) is criticized for "synthetical" nature of tests there. On the other hand, lots of synthetical tests are quite meaningful as well, so I really like to see one more test here. We might try to adopt it for ORMBattle (uff, I really miss 50-hour days ;) ).

psulek wrote:

...I really miss 50-hour days ;) ).

true, true :-)