Question

Hi,

The following example proves that we can not query for objects which have been newly added. The result of the query below gives a count = 0. Is it true that DO4 does not see the objects in the cache? If true, how do we get these (without persisting the data to the DB)?

Regards Paul Sinnema Diartis AG

private static void TestQueryForNewCreated(Domain domain)
        {
            // Opens the Session with DisconnectedState attached and Connect() executed
            ModelContext context = ModelContext.Instance;

            CommunicationKindType type = new CommunicationKindType();

            type.Z_IsDefault = true;

            var q = from comm in Query.All<CommunicationKindType>()
                    where comm.Z_IsDefault
                    select comm;

            var result = q.ToList();
        }

Updated at 13.07.2010 16:41:36

Do you use attached DisconnectedState (this isn't clear from the sample)? If yes, this is correct: we never flush the changes automatically in this case, but queries you run always hit the real DB.

Hi Alex,

You could have read it from the comment in the example :o (// Opens the Session with DisconnectedState attached and Connect() executed)

What about the second question? How do we get them without persisting the data to the DB? If there's no way to get them this will lead to unwanted results. We don't want to persist anything until all data is correct.

Take for instance a real KLIB (our application) example.

Some of our lists use LINQ queries to refresh their content (the query is handed over as a parameter and kept inside the list for later use). This construction will become impossible if the data in the cache is never found. Any Add() would not be noticed by the list. IMHO the Cache should be transparent to LINQ or in other words, any entities added to the DisconnectedState should become visible in LINQ Queries that are executed against it.

Regards Paul Sinnema Diartis AG

Updated at 13.07.2010 19:46:19

Hi Alex,

Well, the code does run, but it only returns the same rows already fetched using a DB query. In the Example below q1 and q2 both return the same amount of rows (in my case each 3 rows). When I don't execute q1 then no rows are returned.

What I also see is that after executing 'new CommunicationKindType();' the Versions collection in the DisconnectedState has a count of zero (0).

private static void TestQueryForNewCreated(Domain domain)
        {
            // Opens the Session with DisconnectedState attached and Connect() executed
            ModelContext context = ModelContext.Instance;

            AbstractEntity.AddStamps = false; // just some code to prevent stamping during SaveChanges()

            CommunicationKindType type = new CommunicationKindType();

            type.Z_IsDefault = true;
            type.Z_NameLanguageString1 = "English";
            type.Z_NameLanguageString2 = "German";
            type.Z_NameLanguageString3 = "French";
            type.Z_NameLanguageString4 = "Italian";

            type.Z_Attribute = "SomeTest";

            type.Z_CreatedBy = 0;
            type.Z_CreatedOn = DateTime.Now;
            type.Z_ChangedBy = 0;
            type.Z_ChangedOn = DateTime.Now;

            var q1 = from comm in Query.All<CommunicationKindType>()
                     where comm.Z_IsDefault
                     select comm;

            var result = q1.ToList();

            var q2 = from comm in context.KLIBSession.DisconnectedState.AllCached<CommunicationKindType>()
                    where comm.Z_IsDefault
                    select comm;

            result = q2.ToList();

            ModelContext.Instance.SaveChanges();

            // Also closes the Session
            ModelContext.Instance.Dispose();
        }

Regards Paul Sinnema Diartis AG

Updated at 14.07.2010 8:20:33

Hi Alex,

I still don't get it completely. Are you telling me that locally added entities don't show up in the DisconnectedState.Versions and there is no way we can get all of them (remote and local together) without persisting the Data to the DB? You must be kidding me. So there's no way we can get all of the cached entities including the freshly added ones without persisting the data to the DB?

IMHO that is a flaw in DO4's architecture. Can you explain to me what the concept is behind this, because I'm totally missing the point.

The cache should be totally transparent for fetches and changes, meaning that any Entities in the cache together with the changes made to the cache (this includes all CRUD actions) are all visible at anytime (without persisting the data to the DB, i.e. totally local). The DisconnectedState is completely useless, to us, when we can not see all CRUD changes in it at any given time. It would mean that in order to see all the changes, any CRUD action should immediatly be persisted to the DB. This will absolutely lead to inconsistent data.

We could maintain a manual transaction and only commit/rollback at times we wanted, but, even with optimistic locking, that would cause many locks in the DB, which in turn would lead to unwanted deadlock situations (coffee-break syndrom).

[Andres Rohr]Or, another solution for us would be if we could call a special 'ApplyChanges()' call that plays back the changes, of the DisconnectedState, to the Session (or Domain cache) without going to the DB. So a LINQ query would see the changes too.

Regards Paul Sinnema Diartis AG.

Updated at 15.07.2010 6:10:49

Alex,

Somehow I've got the feeling that several topics are related. We've got:

the issue with the Batched Queries which now give a problem during the refresh of list: viewtopic.php?f=29&t=6020&start=0. Here we now get a Version mismatch 'Version of entity with key 'CorrespondenceLanguage, (1)' differs from the expected one.'
the issue where new entities don't show up in the DisconnectedState.Versions viewtopic.php?f=29&t=6014
and last but not least we've got the issue you've been working on last night which we call 'Object Reference Changed' issue

Could it be that we get the 'Version mismatch' because of the 'Object Reference Changed' or 'Disconnected.Versions' problem?

Regards Paul

Updated at 15.07.2010 6:47:48

Alex,

On the Version issue. I had added a Guid to the AbstractEntity (our 1 and only HierarchyRoot element). After setting the MergeMode to PreferSource, I got an Exception saying that a conversion from Guid to string was not possible. Maybe a hint on where the problem lies?

Regards Paul

Updated at 15.07.2010 11:11:03

at Xtensive.Storage.DisconnectedState.RegisterState(Key key, Tuple tuple, VersionInfo version, MergeMode mergeMode) at Xtensive.Storage.DisconnectedState.RegisterState(Key key, Tuple tuple) at Xtensive.Storage.Disconnected.DisconnectedSessionHandler.RegisterEntityState(Key key, Tuple tuple) at Xtensive.Storage.Linq.Materialization.ItemMaterializationContext.Materialize(Int32 entityIndex, Int32 typeIdIndex, TypeInfo type, Pair1[] entityColumns, Tuple tuple) at lambda_method(Closure , Object[] , Tuple , ItemMaterializationContext ) at Xtensive.Core.DelegateBindExtensions.<>c__DisplayClassa4.<bind>b9(T2 arg2, T3 arg3) at Xtensive.Storage.Linq.Materialization.MaterializationHelper.<>cDisplayClass41.<Materialize>b__3(Tuple tuple) at System.Linq.Enumerable.WhereSelectEnumerableIterator2.MoveNext() at Xtensive.Core.EnumerableExtensions.<batch>d__201.MoveNext() at Xtensive.Core.EnumerableExtensions.<ApplyBeforeAndAfter>d__281.MoveNext() at System.Collections.Generic.List1..ctor(IEnumerable1 collection) at System.Linq.Enumerable.ToListTSource in C:\KLIB2\trunk\KLIB\KLIBDatabase\ObservableEntityList.cs:line 164 at Diartis.KLIB.Model.CorrespondenceLanguage.get_CorrespondenceLanguageList() in C:\KLIB2\trunk\KLIB\KLIBDatabase\Generated\CorrespondenceLanguage.Generated.cs:line 83 at Diartis.KLIB.KLIBViewModel.VMCorrespondenceLanguage.get_AllVMCorrespondenceLanguageList() in C:\KLIB2\trunk\KLIB\KLIBViewModel\VMCorrespondenceLanguage.Generated.cs:line 43

Updated at 22.07.2010 12:20:31

We still have some problems with this (I think it belongs in this thread). During the ApplyChanges() we now get the following exception:

Entity with Key = 'DossierContactingCause, (-3)' does not exist.

Stacktrace:

>   Xtensive.Storage.dll!Xtensive.Storage.Query.Single(Xtensive.Storage.Session session, Xtensive.Storage.Key key) Line 397 C#
    Xtensive.Storage.dll!Xtensive.Storage.Operations.EntityFieldSetOperation.Execute(Xtensive.Storage.Operations.OperationExecutionContext context) Line 65 + 0xf bytes C#
    Xtensive.Storage.dll!Xtensive.Storage.OperationLog.Replay(Xtensive.Storage.Session session) Line 74 + 0x2d bytes    C#
    Xtensive.Storage.dll!Xtensive.Storage.DisconnectedState.ApplyChanges(Xtensive.Storage.Session targetSession) Line 207 + 0x4c bytes  C#
    Xtensive.Storage.dll!Xtensive.Storage.DisconnectedState.ApplyChanges() Line 184 + 0x28 bytes    C#
    KLIBDatabase.dll!Diartis.KLIB.Model.ModelContext.SaveChanges() Line 363 + 0x30 bytes    C#
    KLIBViewModel.dll!Diartis.KLIB.KLIBViewModel.VMContext.SaveChanges() Line 84 + 0x1c bytes   C#
    KLIBBase.dll!Diartis.KLIB.KLIBContext.SaveChanges() Line 289 + 0x21 bytes   C#
    KLIBBase.dll!Diartis.KLIB.KLIBContext.OnSave(object target, System.Windows.Input.ExecutedRoutedEventArgs e) Line 101 + 0x1c bytes   C#
    PresentationCore.dll!System.Windows.Input.CommandBinding.OnExecuted(object sender, System.Windows.Input.ExecutedRoutedEventArgs e) + 0xe2 bytes 
    PresentationCore.dll!System.Windows.Input.CommandManager.ExecuteCommandBinding(object sender, System.Windows.Input.ExecutedRoutedEventArgs e, System.Windows.Input.CommandBinding commandBinding) + 0xd2 bytes  
    PresentationCore.dll!System.Windows.Input.CommandManager.FindCommandBinding(System.Windows.Input.CommandBindingCollection commandBindings, object sender, System.Windows.RoutedEventArgs e, System.Windows.Input.ICommand command, bool execute) + 0x100 bytes  
    PresentationCore.dll!System.Windows.Input.CommandManager.FindCommandBinding(object sender, System.Windows.RoutedEventArgs e, System.Windows.Input.ICommand command, bool execute) + 0x234 bytes 
    PresentationCore.dll!System.Windows.Input.CommandManager.OnExecuted(object sender, System.Windows.Input.ExecutedRoutedEventArgs e) + 0x3a bytes 
    PresentationCore.dll!System.Windows.RoutedEventArgs.InvokeHandler(System.Delegate handler, object target) + 0x53 bytes  
    PresentationCore.dll!System.Windows.EventRoute.InvokeHandlersImpl(object source, System.Windows.RoutedEventArgs args, bool reRaised) + 0x271 bytes  
    PresentationCore.dll!System.Windows.UIElement.RaiseEventImpl(System.Windows.DependencyObject sender, System.Windows.RoutedEventArgs args) + 0x14e bytes 
    PresentationCore.dll!System.Windows.Input.RoutedCommand.ExecuteImpl(object parameter, System.Windows.IInputElement target, bool userInitiated) + 0x17f bytes    
    KLIBControlLibrary.dll!Diartis.KLIB.KLIBControlLibrary.StandardControls.KLIBTabControl.KLIBTabControlSelectionChanged(object sender, System.Windows.Controls.SelectionChangedEventArgs e) Line 63 + 0x42 bytes  C#

This thread was imported from our support forum. The original discussion may contain more detailed answer.

Answer 1

First of all, the solution: DisconnectedState.Versions allows you to get all pairs of Key and VersionInfo objects it caches. So nearly this method will return all the objects of type T cached in DisconnectedState:

public IEnumerable<T> AllCached<T>(this DisconnectedState disconnectedState)
  where T: class, IEntity
{
  var keys = disconnectedState.Versions.Select(pair => pair.Key);
  var entities = 
    from key in keys
    let entity = Query.SingleOrDefault<T>(key)
    where entity!=null && !entity.IsRemoved
    select entity;
  return entities;
}

I didn't tested it, so there might be non-essential mistakes. That's the point you must (or can) use in this case instead of Query.All<t>(). This implies such "queries" will be evaluated via LINQ to IEnumerable, but this should be acceptable for relatively small data sets you have on the client.

I also should explain why we can't do this automatically for queries, when DisconnectedState is attached: it is much more that just "merging" the results of local and remote queries. Results of many remote operations simply can't be "merged" to produce the right final result - e.g. this can't be done in case there is any aggregation or .GroupBy(). As far as I can judge, the only way to produce the right result is to perform almost the whole calculation on the client (= fetch almost the whole database and "crunch" the data here), which is definitely not what you would expect.

So we decided to do what we really can do well: run the query remotely and bring all the data to the client. It will be merged into DisconnectedState, and exposes with the changes it caches. But these changes won't affect on remote query result. At least, the behavior we have here is fully clear.

See also: viewtopic.php?f=29&t=6011#p15187

Yes, suggested method should return only the objects already fetched into the DisconnectedState, since it relies on info about versions of Entities cached in DisconnectedState.

So there is no silver bullet: you have two storages, one is actually a "diff" above another, i.e. ActualStorage = DisconnectedStorage.ApplyChangesTo(RemoteStorage). Two storages on this schema are real (DisconnectedStorage and RemoteStorage), but one is virtual represenation we provide for you. And you'd like to run a query over ActualStorage, that actually doesn't exist - so to run a query there, we must either build its full view locally (not acceptable), or transform RemoteStorage to it (by applying the changes). There is no other way to run a query over this storage.

Some simplest queries (e.g. using just filtering) can be executed in "remote query + local refinement" fashion, but generally that's a case you can implement by your own. LiveQuery described in another post targets exactly this problem: run the query once (e.g. remotely), bind the result to UI and see how it is changing when you modify the entities locally.

> Are you telling me that locally added entities don't show up in the DisconnectedState.Versions

Hmm... They should. I misunderstood you - I though you're trying to get them via Query.All<t>().

Probably, that's a bug here (i.e. only fetched versions are exposed there) - I gave you the code, but I didn't try it myself. I'll check this today.

> The cache should be totally transparent for fetches and changes, meaning that any Entities in the cache together with the changes made to the cache

That's exactly the case we have. Of course, except direct LINQ queries to the storage - they work remotely, and thus return result based on remote data, although we expose all the entities returned by such queries already with your modifications.

> [Andres Rohr]Or, another solution for us would be if we could call a special 'ApplyChanges()' call that plays back the changes, of the DisconnectedState, to the Session (or Domain cache) without going to the DB. So a LINQ query would see the changes too.

Actually, you already can do this:

Open one more Session
Open a transaction there
Replay a copy of operation log there
Run your query, serialize the keys of entities you got // If this suits for you; shortly we'll allow you to serialize generally any graph with entities in "by ref" mode
Rollback the transaction.

The last issue isn't fixed yet - needs further evaluation.

But:

DisconnectedState now implements IEnumerable<entity>
We added DisconnectedState.All<t>() method returning sequence of all the entities of the specified type it caches (except removed ones, of course - although this info is cached as well).
DisconnectedState.GetPersistenceState(Key key) and DisconnectedState.AttPersistenceStates() were added
Now Session is capable of remapping Entity keys cached inside it. OperationLog uses this feature on Replay; you can invoke it manually via DirectStateAccessor.Get(session).RemapEntityKeys(KeyMapping keyMapping) method.

The cause is:

when top-level operation is being logged by DisconnectedState (DS), nested ones aren't (we intentionally prevent this, since the topmost operation, when replayed, must lead to execution of the whole chain)
but in your case a new entity is created inside it (as result of event handler invocation), and we don't log this operation; so DO can't figure out how to deal with further updates of properties of newly created entity: it doesn't know how it was created (no such entry in log), and thus can't associate a new entity with old key. Imagine: When DS is attached:
- Create Entity #-1 -> Logged
- Create Entity #-2 -> not logged, since top-level operation still continues, and its reply will anyway lead to this one
- Update Entity #-1 -> Logged
- Update Entity #-2 -> Logged
When replaying this, the following happens:
- Create Entity #-1, new key is #100, remember it for further key remapping
- Create Entity #101 - this operaton is "induced" by the above one, nothing is logged in key remapping list
- Update Entity #-1 - key is remapped to #100, because there is info for this, operation completes successfully.
- Update Entity #-2 -> key isn't remapped, since there is no info for this. FAIL. Btw, strange we didn't implement support for this yet. I.e. the case is known for me, I immediately remembered we had special support for this in v3.X - there was Offline layer operating nearly the same way.

The solution we'll use here will be nearly the same - i.e. you should give identifying names to entities created inside topmost operations:

var newCustomerA = ...;
newCustomerA.IdentifyAs("CustomerA");
var newCustomerB = ...;
newCustomerB.IdentifyAs("CustomerB");
var newOrder = ...; // No .IdentifyAs, so it will be automatically uniquely identified in scope of current operation, e.g. as "Entity#1"

These names will be used to establish mappings between old and new keys of same named entities.

Updated at 13.07.2010 16:41:36

Updated at 13.07.2010 19:46:19

Updated at 14.07.2010 8:20:33

Updated at 15.07.2010 6:10:49

Updated at 15.07.2010 6:47:48

Updated at 15.07.2010 11:11:03

Updated at 22.07.2010 12:20:31

Subscription:

Related questions