Memory usage changes?

[Dave Britten]

Has Free42 and/or the DM42 changed its memory usage behavior within the past year or so? My usual loadout of programs and data used to report somewhere around 24,000-30,000 bytes free, whereas now it seems to be around 8,000.

The weird thing is I have an old .s42 state file that's 38K, and the DM42 reports 34,592 bytes free when loaded, whereas my current .f42 state file is only 26K, but reports 7,976 bytes free when loaded. I've had "disappearing memory" issues on my 48 due to corrupt/invisible objects before - not sure if Free42 is susceptible to anything similar.

EDIT: I just installed the latest firmware (I was one release behind), and I still see 7,976, so no change there.

[Thomas Okken]

In Free42, the only things that use a lot of memory are large matrices and memory fragmentation (and memory fragmentation only eats into available memory if you're working with large matrices). Try deleting all variables, clearing the stack and LASTx, and then saving the state and rebooting. If available memory is still low after that, SwissMicros may need to look into this. On the Free42 side, there have been no changes to the way it allocates memory in ages.

[Dave Britten]

In Free42, the only things that use a lot of memory are large matrices and memory fragmentation (and memory fragmentation only eats into available memory if you're working with large matrices). Try deleting all variables, clearing the stack and LASTx, and then saving the state and rebooting. If available memory is still low after that, SwissMicros may need to look into this. On the Free42 side, there have been no changes to the way it allocates memory in ages.

Okay, I do work with matrices for data logging purposes, though I'm not sure they would be considered large (usually less than 20 elements). I'll try deleting a few that I don't need, and if that doesn't help, I'll use the Windows Free42 to export the ones I need to keep. Thanks Thomas.

[Thomas Okken]

20 elements, that doesn't really add up... at 16 bytes per element (32 for complex matrices) plus a bit of overhead, you'd have to have much larger ones to account for 20 missing kilobytes...

[Dave Britten]

Alright, there's definitely something squirrelly going on. I have a state file from an older release named "temp.s42", which is 38K in size. Right after I load it, MEM reports 34,592 bytes free. I then immediately save this as a new state file, say "temp3.f42", which ends up being 25K. After saving and reloading this new state file, I see 8,000 bytes free.

[Thomas Okken]

That sounds like a DM42 issue. The new state file being a bit smaller makes sense, since the new state file format introduced in 2.5 is a bit more efficient than the original one. But free memory dropping by that much is not OK. The Free42 core deletes its variables when it unloads a state so there shouldn't be memory leakage when switching state files.

[Dave Britten]

That sounds like a DM42 issue. The new state file being a bit smaller makes sense, since the new state file format introduced in 2.5 is a bit more efficient than the original one. But free memory dropping by that much is not OK. The Free42 core deletes its variables when it unloads a state so there shouldn't be memory leakage when switching state files.

Very weird. Has Free42 ever saved amount of free memory in the state file? I'm wondering if the older DM42 firmware had bugs with calculating free memory, and that ~30K free figure was bogus. Still, seems odd that the current state file size + free memory don't add up to anywhere near 70K.

If you or anybody from SM want copies of the state files to autopsy, just PM me an email address to send them to.

[Thomas Okken]

I'm guessing that the amount of free memory reported by the DM42 now is bogus, or maybe it's reporting the size of the memory segment that the heap hasn't yet expanded into or something. You can check by trying to create large matrices -- 64x1 takes up a bit over 1 kilobyte, so if you are able to allocate, say, a 640x1 matrix, 10 kilobytes, it sounds like that should fit despite MEM reporting that not that much memory is available.

The state file wouldn't tell me anything, that's just a sequence of data objects and programs. I bet David can shine light on this without even firing up a debugger.

[Dave Britten]

With 12,296 bytes free, creating a 640x1 matrix on the stack takes it down to 1,352 bytes free. Attempting to create a 480x2 (after deleting the 640x1 matrix) reports "Insufficient Memory". The plot thickens...

[Thomas Okken]

That suggests that MEM is telling the truth, at least as far as the maximum possible single allocation is concerned. To distinguish between memory fragmentation and a memory leak, you could try allocating a bunch of smaller matrices and check if they cause MEM to decrease as expected.

Note that merely duplicating a matrix does not cause a new block of data to be allocated, since the original and the duplicate will share memory initially, but NEWMAT always allocates a brand new matrix.