Question

Consider this (partial) model:

[Index("Text", Unique = true)]
public class UniqueText: Entity
{
...
        [Field]
        public string Text { get; set; }
}

I have multiple threads and lists of strings that should be stored, if the particular string don't exist. Currently i have a ugly bad implementation for this that just checks for every string if it exists and add it if it don't.

The string sets -to check and to insert - are quite large (e.g. 8000 strings at once).

I would like to do something like this for performance reasons: Get all strings that don't exist in one batch - insert all entities with new strings in one batch.

I have multiple worker threads and sometimes they have the same string which should be inserted. To two inserts with the same string are executed. One of them fail with a unique constraint violation. How can this be avoided / handled best?

Accepted Answer

Hello Thomas,

I'd start with the following pattern:

// a set of strings to store
var strings;

using (var s = Domain.OpenSession()) {
  using (var t = s.OpenTransaction()) {

    // This query creates a temporary table
    // and fill it with strings to store
    // As a result, it returns strings that already exist
    var existing = s.Query.All<TextEntity>()
      .Where(i => i.Text.In(strings))
      .Select(i => i.Text);

    // Creating entities for strings that don't exist
    foreach (var text in strings.Except(existing))
      new TextEntity { Text = text };

    t.Complete();
  }
}

Hope that helps.

Subscription:

Related questions