Hi, I've got the feeling the FutureQueries are not well documented. The only documentation I find is in the Manual and this is not extensive. In the Reference Guide we can't even find the Methods (Execute(), ExecuteFuture(), etc..). There are no Future Queries in the Samples and the Wiki. If there is any documentation then I've missed it. Any pointers here would be valued. We discovered for instance that the ExecuteFuture() only works when a transaction is active. If this interpretation is true and there is no other way to execute them (i.e. without an open transaction) the future queries can not be used in our scenario. We centrally open a Session, add a DisconnectedState and call it's Connect(). We would like to use the Future Queries to preload certain tables (reference tables) at the start of the application. Future Queries would be the perfect means to reach that goal, but since we totally rely on auto-transactions, no transaction is open from the start of the program and Future Queries will fail. We also tried opening our own transaction but as soon as that is Disposed, the Future Queries that were open inside transaction, cannot be accessed from outside the transaction. Ok, here are some questions:
Regards Updated at 27.07.2010 12:33:03Just found the following link (via Google) http://help.dataobjects.net. There the Methods are described but still not extensively. Updated at 29.07.2010 10:06:04Hi Alex, I still think we don't fully understand the concept of DO. Everything in DO seem transaction based. This would mean that there are no 'disconnected' Entities. Any change made to an Entity is always persisted to the DB. We interpreted the DisconnectedState as a means to create disconnected recordsets. In a way that's what they are but in another way they aren't. If we want to use the full fledged functionality of DO, we need to open a transaction. What's really unclear is how the DisconnectedState and (nested) transaction cooperate. Questions like:
are not really answered in the Documentation.
I could think of a lot more questions, but the basic question is 'What are the concepts behind DO?'. Regards Updated at 29.07.2010 10:15:38Aha.. my colleaque Marco just found out that Future Queries should be executed inside a transaction but we can access the objects read from them outside the transaction. Pfeww that makes life easier. Now we can generate code that preloads all the reference tables in just 1 batch. That's great. This thread was imported from our support forum. The original discussion may contain more detailed answer. |
Alex (Xtensive) wrote:Just checked this - true, .CHM reference from .NET 4.0 version has a set of strange lacks (mainly, absence of member documentation). Almost 100% sure this is a result of some Sandcastle bug, since... The good news is that reference from .NET 3.5 version seems fully correct. Right now I'm updating it to downloads section. .NET 3.5 version is ~99.9% identical to .NET 4.0 version by its members, so please replace its .CHM file by that one. Hopefully, I'll be able to implement its automatic inclusion to installer shortly (the problem is that actually these versions reside in different branches - so from the point of current build process they have nothing common)... Alex (Xtensive) wrote:Concerning future queries: the example form Manual project: http://goo.gl/0iMh You can't use them without a transaction - we create internal query tasks, when such queries are built, and these tasks are executed inside the next batch sent to server, but batches can't cross transaction boundaries. Normally it's even wrong to execute a set of correlated queries without a transaction: you expect the consistent result there (i.e. if first query fetches Authors and second - Books, they must properly relate to each other; now imagine query sent in between simply deletes all the Books), but there is no way to ensure this without a transaction. Concerning your case: I don't understand what prevents you from creating a transaction manually to run a set of queries in batch - can you explain this? Alex (Xtensive) wrote:Let's start from simple questions: > Marco just found out that Future Queries should be executed inside a transaction but we can access the objects read from them outside the transaction. That's true. The transaction is necessary here just to ensure the data is fetched as part of one logical operation. > What happens when we open a transaction. Does it open 1 in the DB too? I know you're talking about the case when DisconnectedState (DS further) is attached to Session, and moreover, it is already "connected". So:
So in short, it's ok if you open many transactions in DS. Actual transactions in DB will run only when you really access the DB inside these transactions. > Can we keep a transaction open accross ApplyChanges() borders? Isn't that going to produce locks in the DB? Yes, you can. Locks will be acquired as they should - DO isn't responsible for this, this is solely a problem of SQL Server. But if you want to release X-locks acquired during ApplyChanges, you must complete the transaction (completing the transaction is the only way to release the locks in DB). Note that ApplyChanges creates either nested or outermost transaction while it runs, and commits/rolls back it dependently on result of the whole operation. Alex (Xtensive) wrote:I also want to answer the question about asynchronous loading\fetches - I remember there was a question touching this. Imagine we have a Session (usSession) with attached DisconnectedState (uiDS); UI is bound to entities provided by this Session. And now we'd like to load a large set of entities into this Session - asynchronously. To achieve this, you must:
That's it. Alex (Xtensive) wrote:Forgot to add: if set to load asynchronously is really large (e.g. 100K...1M entities - I know it's a bad idea to cache this set on the client, so this is just FYI), I'd even split the whole operation to a set of smaller "page load" operations, where each page includes ~ 1K...10K entities. I.e. the described routine must be used sequentially in this case. LeonardiMarco wrote:
That's true. The transaction is necessary here just to ensure the data is fetched as part of one logical operation. But now we have the following problem:
Outside of the transaction I want to foreach all entities:
But on the persons.GetEnumerator() I got the follow Excpetion: "InvalidOperationExcpetion occured: The current transaction is different from the transaction bound to this instance." I don't understand why he needs transaction, beacuse he already has the data from the DB. You agree? Regards Marco Leonardi psulek wrote:
No, it does not have data for persons from DB because you create just IEnumerable<person> persons, which does not load data from DB until it is enumerated or called ToList() like you does with orgnaization.ToList. Correction:
As you can see here i add line personList = persons.ToList(); which iterate through IEnumerable and loads data from DB to generic list List<person> and outside transaction you can iterate variable personList without active transaction. Alex (Xtensive) wrote:Btw, that's why you get IEnumerable<t> as result of Query.ExecuteFuture<t>(() => ...):
Let's compare this to compiled query (Query.Execute<t>(() => ...)):
Caching key is MethodInfo of delegate you pass to Query.Execute<t>(() => ...) - if you write it as anonymous delegate, its MethodInfo is unique in each case, i.e. that's exactly what is necessary here. The delegate itself is invoked just once. Finally, it worth to mention once more that we never materialize the whole result at once. Instead we materialize it in batches; batch size grows up as power of 2 from ~ 16 to 1024, and stops growing. This, in combination with weak reference-based entity state cache, allows DataObjects.Net to browse potentially infinite result sets. LeonardiMarco wrote:As you can see here i add line personList = persons.ToList(); which iterate through IEnumerable and loads data from DB to generic list List<person> and outside transaction you can iterate variable personList without active transaction. Psulek, your above conclusion is not right: If you apply ToList also on persons and you want to iterate it outside of the transactionit throws the same Excpetion. This confirms the statement of Alex: " when you enumerate the enumerable, we either materialize already fetched result (if it's there at the moment of materialization), or force execution of prepared query batch to ensure the result is available " The solution of my problem is to use the List wicht returns the IEnumerable<person>.ToList(). This list must never refresh/update/ to the DB beacuse during the whole programm execution this list does never change. (maybe persons was not the right example, rather a list which fills a ComboBox). Regards Marco Leonardi psulek wrote:I dont understand you, because i write that you must use: LeonardiMarco wrote:
Hi psulek, What I mean is follow: Requires a transaction:
Doesn't require a transaction:
LeonardiMarco wrote:I have the follow problem:
The first foreach which iterates the 'pCodes' takes the entities from the cache and needs no other connection to the DB. But the second foreach (situated outside of the transaction) which iterates the 'pCountrys' executes for each item in the list an SQl query, although he already got the data. Why he doesn't take the data from the cache? Outside of this transaction I'm using 'DisconnectedState'. Hope you can help me Regards Marco Leonardi Alex (Xtensive) wrote:> But the second foreach (situated outside of the transaction) which iterates the 'pCountrys' executes for each item in the list an SQl query, although he already got the data. This may happen only if DisconnectedState isn't attached to the current Session. And one more note: you run the code after calling TransactionScope.Complete() - that's wrong: in our case TransactionScope.Complete just sets the flag that commit must happen instead of rollback, i.e. everything is ok, and thus it's safe to throw an exception from IDisposable.Dispose (see the end of this post, there is an explanation). So TransactionScope.Complete must normally be called right before leaving corresponding using block. LeonardiMarco wrote:
Thats right, the DisconnectedState was not yet connected to the current Session. I have to open the transacation also after the Connect of the DisconnectedState. Thanks for your help Regards Marco Leonardi Yes, exactly. Hi Alex,
This got snowed under a bit. Maybe on your blog or in the Wiki? Regards Yes, I'll do this quite soon. After finishing the fixes I'm busy with right now... Yes, in case with future queries transaction should be anyway opened. |