Alex (Xtensive) wrote:

Just checked this - true, .CHM reference from .NET 4.0 version has a set of strange lacks (mainly, absence of member documentation). Almost 100% sure this is a result of some Sandcastle bug, since...

The good news is that reference from .NET 3.5 version seems fully correct. Right now I'm updating it to downloads section.

.NET 3.5 version is ~99.9% identical to .NET 4.0 version by its members, so please replace its .CHM file by that one. Hopefully, I'll be able to implement its automatic inclusion to installer shortly (the problem is that actually these versions reside in different branches - so from the point of current build process they have nothing common)...

Alex (Xtensive) wrote:

Concerning future queries: the example form Manual project: http://goo.gl/0iMh

You can't use them without a transaction - we create internal query tasks, when such queries are built, and these tasks are executed inside the next batch sent to server, but batches can't cross transaction boundaries. Normally it's even wrong to execute a set of correlated queries without a transaction: you expect the consistent result there (i.e. if first query fetches Authors and second - Books, they must properly relate to each other; now imagine query sent in between simply deletes all the Books), but there is no way to ensure this without a transaction.

Concerning your case: I don't understand what prevents you from creating a transaction manually to run a set of queries in batch - can you explain this?

Alex (Xtensive) wrote:

Let's start from simple questions:

> Marco just found out that Future Queries should be executed inside a transaction but we can access the objects read from them outside the transaction.

That's true. The transaction is necessary here just to ensure the data is fetched as part of one logical operation.

> What happens when we open a transaction. Does it open 1 in the DB too?

I know you're talking about the case when DisconnectedState (DS further) is attached to Session, and moreover, it is already "connected". So:

• When you open a transaction, nothing happens in DB, but logical transaction is opened in DS. This means you can change something in entities and then roll back it - the state will be rolled back.

• DB-level transaction starts when the first operation hitting DB is executed by DS. E.g. query.

• DB-level transaction gets closed when the transaction in DS gets closed.

So in short, it's ok if you open many transactions in DS. Actual transactions in DB will run only when you really access the DB inside these transactions.

> Can we keep a transaction open accross ApplyChanges() borders? Isn't that going to produce locks in the DB?

Yes, you can. Locks will be acquired as they should - DO isn't responsible for this, this is solely a problem of SQL Server. But if you want to release X-locks acquired during ApplyChanges, you must complete the transaction (completing the transaction is the only way to release the locks in DB).

Note that ApplyChanges creates either nested or outermost transaction while it runs, and commits/rolls back it dependently on result of the whole operation.

Alex (Xtensive) wrote:

I also want to answer the question about asynchronous loading\fetches - I remember there was a question touching this.

Imagine we have a Session (usSession) with attached DisconnectedState (uiDS); UI is bound to entities provided by this Session. And now we'd like to load a large set of entities into this Session - asynchronously. To achieve this, you must:

• create a background thread

• create its own Session (bgSession) and its own DisconnectedState (bgDS) attached to it

• run the queries fetching the data to bgSession. Since it has attached DS, all the entity states fetched by these queries will be cached by bgDS. Note that you must enumerate the query result (e.g. using our IEnumerable<t>.Run() extension method from Xtensive.Core) to ensure it is fully materialized - DO delays nearly any operation until the data is really necessary, so if this isn't done, nothing will be cached.

• detach bgDS, dispose bgSession

• pass bgDS to the UI thread

• call uiDS.Merge(bgDS). This operation must be synchronous; also note MergeMode can be passed to Merge method (if it is not specified, uiDS.MergeMode will be used), so merge conflicts are resolved at this stage. You can get an exception, if mergeMode==MergeMode.Strict, and there is a version conflict for some entity (i.e. its version cached in uiDS differs from version in bgDS).

That's it.

Alex (Xtensive) wrote:

Forgot to add: if set to load asynchronously is really large (e.g. 100K...1M entities - I know it's a bad idea to cache this set on the client, so this is just FYI), I'd even split the whole operation to a set of smaller "page load" operations, where each page includes ~ 1K...10K entities.

I.e. the described routine must be used sequentially in this case.

LeonardiMarco wrote:

Marco just found out that Future Queries should be executed inside a transaction but we can access the objects read from them outside the transaction.

That's true. The transaction is necessary here just to ensure the data is fetched as part of one logical operation.

But now we have the following problem:

using (Transaction trans = Transaction.Open(session)
{
   IEnumerable<Person> persons = Query.ExecuteFuture<Person>(() => .....
   IEnumerable<Organization> orgnaization = ...
   ...
   orgnaization.ToList
   trans.Complete();
}

Outside of the transaction I want to foreach all entities:

using (IEnumerator<T> en = persons.GetEnumerator())
                {
                    while (en.MoveNext())
                    {
                        Add(en.Current);
                    }
                }

But on the persons.GetEnumerator() I got the follow Excpetion:

"InvalidOperationExcpetion occured: The current transaction is different from the transaction bound to this instance."

I don't understand why he needs transaction, beacuse he already has the data from the DB.

You agree?

Regards Marco Leonardi

psulek wrote:

... I don't understand why he needs transaction, beacuse he already has the data from the DB.

No, it does not have data for persons from DB because you create just IEnumerable<person> persons, which does not load data from DB until it is enumerated or called ToList() like you does with orgnaization.ToList.

Correction:

List<Person> personList;
using (Transaction trans = Transaction.Open(session)
{
   IEnumerable<Person> persons = Query.ExecuteFuture<Person>(() => .....
   personList = persons.ToList();
   trans.Complete();
}

As you can see here i add line personList = persons.ToList(); which iterate through IEnumerable and loads data from DB to generic list List<person> and outside transaction you can iterate variable personList without active transaction.

Alex (Xtensive) wrote:

Btw, that's why you get IEnumerable<t> as result of Query.ExecuteFuture<t>(() => ...):

• we queue the query for future execution when you run Query.ExecuteFuture - nothing like this happens with regular compiled query;

• when you enumerate the enumerable, we either materialize already fetched result (if it's there at the moment of materialization), or force execution of prepared query batch to ensure the result is available.

Let's compare this to compiled query (Query.Execute<t>(() => ...)):

• Invocation of this method simply looks up for query compilation result in cache (+ compiles the query and adds it, if nothing is cached), and binds it with enumerable it returns;

• Enumeration of this enumerable leads to actual query parameterization and execution - as with regular LINQ query. The only difference is that translation stage isn't necessary here.

Caching key is MethodInfo of delegate you pass to Query.Execute<t>(() => ...) - if you write it as anonymous delegate, its MethodInfo is unique in each case, i.e. that's exactly what is necessary here. The delegate itself is invoked just once.

Finally, it worth to mention once more that we never materialize the whole result at once. Instead we materialize it in batches; batch size grows up as power of 2 from ~ 16 to 1024, and stops growing. This, in combination with weak reference-based entity state cache, allows DataObjects.Net to browse potentially infinite result sets.

LeonardiMarco wrote:

As you can see here i add line personList = persons.ToList(); which iterate through IEnumerable and loads data from DB to generic list List<person> and outside transaction you can iterate variable personList without active transaction.

Psulek, your above conclusion is not right: If you apply ToList also on persons and you want to iterate it outside of the transactionit throws the same Excpetion. This confirms the statement of Alex: " when you enumerate the enumerable, we either materialize already fetched result (if it's there at the moment of materialization), or force execution of prepared query batch to ensure the result is available "

The solution of my problem is to use the List wicht returns the IEnumerable<person>.ToList(). This list must never refresh/update/ to the DB beacuse during the whole programm execution this list does never change. (maybe persons was not the right example, rather a list which fills a ComboBox).

Regards Marco Leonardi

psulek wrote:

I dont understand you, because i write that you must use:
personList = persons.ToList() where variable persons is of type IEnumerable<person> and you tell me that this is not good but your solution is to use IEnumerable<person>.ToList(). I think this is same as my solution, right?

LeonardiMarco wrote:

I dont understand you, because i write that you must use:
personList = persons.ToList() where variable persons is of type IEnumerable<person> and you tell me that this is not good but your solution is to use IEnumerable<person>.ToList(). I think this is same as my solution, right?

Hi psulek, What I mean is follow:

Requires a transaction:

IEnumerable<Person> persons = Query.ExecuteFuture<Person>(() => .....
using(Transaction trans = TranscationScope.Open(..)
{
   persons.ToList(); // data is loaded from DB
   trans.Complete
}
 using (IEnumerator<T> en = persons.GetEnumerator()) // Would raise an exception
{
    while (en.MoveNext())
    {
       Add(en.Current);
     }
}

Doesn't require a transaction:

List<Person> personList;
IEnumerable<Person> persons = Query.ExecuteFuture<Person>(() => .....
using(Transaction trans = TranscationScope.Open(..)
{
   personList = persons.ToList(); // data is loaded from DB
   trans.Complete
}
foreach(Person pers in personList) // doesn't require a transaction
{
    Console.Writeline(pers.Firstname);
}

LeonardiMarco wrote:

I have the follow problem:

List<PostalCode> pCodes;
List<Country> pCountrys;

using (TransactionScope trans = Transaction.Open(session))
{
    pCodes = Query.All<PostalCode>().ToList();
    pCountrys = Query.All<Country>().ToList();

    trans.Complete();

    foreach (PostalCode pc in pCodes)
    {
                Console.WriteLine(pc.IdP.Value);
    }
}

foreach (Country co in pCountrys)
{
    Console.WriteLine(co.IdP.Value);
}

The first foreach which iterates the 'pCodes' takes the entities from the cache and needs no other connection to the DB. But the second foreach (situated outside of the transaction) which iterates the 'pCountrys' executes for each item in the list an SQl query, although he already got the data. Why he doesn't take the data from the cache?

Outside of this transaction I'm using 'DisconnectedState'. Hope you can help me

Regards Marco Leonardi

Alex (Xtensive) wrote:

> But the second foreach (situated outside of the transaction) which iterates the 'pCountrys' executes for each item in the list an SQl query, although he already got the data.

This may happen only if DisconnectedState isn't attached to the current Session.

And one more note: you run the code after calling TransactionScope.Complete() - that's wrong: in our case TransactionScope.Complete just sets the flag that commit must happen instead of rollback, i.e. everything is ok, and thus it's safe to throw an exception from IDisposable.Dispose (see the end of this post, there is an explanation).

So TransactionScope.Complete must normally be called right before leaving corresponding using block.

LeonardiMarco wrote:

This may happen only if DisconnectedState isn't attached to the current Session.

Thats right, the DisconnectedState was not yet connected to the current Session. I have to open the transacation also after the Connect of the DisconnectedState.

Thanks for your help

Regards Marco Leonardi