I am in the process of migrating a legacy binary file format. Part of this involves converting objects to a byte array. But after a while I run into an OutOfMemoryException. I created a small sample program to demonstrate the problem. You can download it here.

for (int i = 0; i < 20; i++)
    Training training = service.GetNewTraining();
    training.Name = Guid.NewGuid().ToString();

    for (int j = 0; j < 10; j++)
        Run run = service.GetNewRun();

        run.Created = DateTime.Now;
        run.Distance = GetRandom(10000, 40000); // Random between 10k and 40k.

        TimeSpan time = TimeSpan.FromHours((run.Distance / 1000) / 25.0);
        for (int k = 0; k < time.TotalSeconds; k++)
            RunDataSample sample = new RunDataSample
                Cadence = GetRandom(60, 110),
                Distance = (float)(time.TotalHours * 25),
                Time = TimeSpan.FromSeconds(k),
                HeartBeat = GetRandom(80, 180),
                Speed = 25,
                Power = GetRandom(200, 400)




I once tried adding a GC.Collect() call after the service.Save() call, but that didn't help either.

I have no need for the objects to be in the memory after I saved them. How can I clean them up properly?

In the included sample I am using an in-memory database, but in my real application I am using an SqlCe database with an x86 build.

asked May 11 '11 at 09:43

jensen's gravatar image


edited May 11 '11 at 10:06

Hello jensen, I'll check this.

(May 11 '11 at 10:21) Dmitri Maximov Dmitri%20Maximov's gravatar image

I want to mention I am going to make a change to the data model as used in the sample application. I do not need the entire collection of samples each time, so I will only initialize the exposed collection when it's first accessed. When switching screens the object is not required, so it should be disposed and I will re-get it from the database when required. Since I collect 25 samples which I basically need on 1 single screen a user will almost never access there's no point in initializing them with the OnInitialize override.

But that won't solve the import OOME I'm having now.

(May 12 '11 at 04:29) jensen jensen's gravatar image

I did some test with manually creating the session, open the session, open the transaction each time and that worked perfectly. I timed it and I get about 2 minutes to complete the operation. When using auto activation it takes about 5 minutes before the exception is thrown.

Now, I really would like to use the auto activation, because it would be a real pain if I need to manually manage the session each time. In a WPF app this is really difficult.

(May 12 '11 at 06:07) jensen jensen's gravatar image

Something else that's bothering me. According to the documentation of Session.Persisting it will be called "when Session is about to SaveChanges() changes.". And SaveChanges() tells me I need to call it manually, which I do.

But the persisting event is thrown each and every time something in the object changes, even if it's not a property that has to persist. (In this case the collection.) How can I work around that?

(May 12 '11 at 08:22) jensen jensen's gravatar image

I tried setting the collection to null after each Run was saved, and that helped a bit. No more OOME, but I still have the idea that the entire entity "hangs" on the Session, which is not needed. But I don't want to do that every time I use an object. If it's not in use anymore (when I switch screens) it should get disposed automatically.

I am also convinced the documentation of SaveChanges is wrong. If I use the DisconnectedState option, the Persisting event is still called every time a [Field] changes.

(May 16 '11 at 09:48) jensen jensen's gravatar image


sorry for the delay, I'll try investigating the issue tomorrow. Thanks for your patience.

(May 16 '11 at 09:49) Dmitri Maximov Dmitri%20Maximov's gravatar image

I have a similar problem. Any ETA on this?

(May 17 '11 at 04:10) TimM TimM's gravatar image

One Answer:

Hello jensen,

  1. To import a huge amount of data I strongly recommend you to use SessionOptions.ServerProfile & manually open and complete transactions. This will boost the entire import process drastically.

  2. In case you definitely want to do the import with SessionOptions.ClientProfile then reopen sessions from time to time, as Session in SessionOptions.ClientProfile doesn't clean its caches at all because it is not intended to be used with that amount of data.

  3. Session.Persist in your case is called so frequent because you use auto transactions (which are used intentionally in SessionOptions.ClientProfile), so every setting a persistent property or adding an item to EntitySet results into commit of a small auto transaction, which in turn lead to Persist method call. To avoid it, open and commit the transactions manually.

BTW, I've updated the sample and add a transaction for each step of the outer for loop (20 iterations), as a result, I got only 20 Persist calls.

answered May 17 '11 at 10:11

Dmitri%20Maximov's gravatar image

Dmitri Maximov

I can do that for the importer, but what if I'm unable to frequently re-open a session during the run of my application?

I have a few services which are initialized at the start of the application and which should exist for the entire run of the application. These services contain some objects from the database like the current player, currently selected training and so forth. If I need to re-open the database I lose the current object and need to get all of those again, increasing complexity of my application, if it would be possible at all.

(May 17 '11 at 10:24) jensen jensen's gravatar image

I also have a screen which provides an overview of all runs. You can select one, which loads the byte[] of samples into the memory. When you close the screen and go back to the overview I perform a new query, but as far as I understand all the loaded bytes will still be kept in memory, attached to the session. Or will it somehow be cleaned?

If I understand correctly, I still would run into the same exception when analyzing a lot of different runs.

(May 17 '11 at 10:25) jensen jensen's gravatar image

The case with the long running services and overview screen are OK, they won't consume lot's of data. As for the screen that loads byte[] of samples, I'd open it in a new Session and dispose it after the screen is closed.

(May 17 '11 at 10:39) Dmitri Maximov Dmitri%20Maximov's gravatar image
Your answer
Please start posting your answer anonymously - your answer will be saved within the current session and published after you log in or create a new account. Please try to give a substantial answer, for discussions, please use comments and please do remember to vote (after you log in)!
toggle preview

powered by OSQA