Note: This discussion is about an older version of the COMSOL Multiphysics® software. The information provided may be out of date.

Discussion Closed This discussion was created more than 6 months ago and has been closed. To start a new discussion with a link back to this one, click here.

How to save large fem structure into .mat file

Please login with a confirmed email address before reporting spam

Hi, all,

When I tried the 'save filename.mat fem' command in COMSOL with Matlab R2010a, the error is as below:

java.lang.OutOfMemoryError:Java heap space

Then I changed the default heap size parameters from 256m (default) to 2g.

I also tried -v6 -v7 -v7.3 before the command line.
The size of the fem structure is about 2g.

I even tried to divide the structure, but it only works for scalar structure.


Since none of these works, I still could not save the fem structure in my simulation.

If you have any ideas or comments, it will be greatly appreciated.

Here is my fem structure looks like:


version: [1x1 struct]
meshtime: 'TIME'
appl: {[1x1 struct] [1x1 struct] [1x1 struct]}
sdim: {{1x2 cell} {1x2 cell} {1x2 cell}}
frame: {'mesh' 'xy' 'ale'}
shape: {1x13 cell}
gporder: {[4] [30] [2]}
cporder: {[2] [1]}
sshape: {[1x1 struct] [1x1 struct] [1x1 struct] [1x1 struct]}
border: 1
form: 'weak'
equ: [1x1 struct]
bnd: [1x1 struct]
pnt: [1x1 struct]
var: {1x64 cell}
elemmph: {1x11 cell}
eleminitmph: {[1x1 struct]}
const: {1x12 cell}
draw: [1x1 struct]
geom: [1x1 geom2]
mesh: [1x1 femmesh]
xmesh: [1x1 com.femlab.xmesh.Xmesh]
sol: [1x1 femsol]




10 Replies Last Post Dec 12, 2011, 5:27 p.m. EST

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago Nov 16, 2011, 12:02 p.m. EST
This week I tired to save another structure, the size is about 400Mb, it gave me error again:
??? Error using ==> save
Java exception occurred:
java.lang.OutOfMemoryError: Java heap space

My workstation configuration: wins xp 64bit 32G RAM


memory
Maximum possible array: 21408 MB (2.245e+010 bytes) *
Memory available for all arrays: 21408 MB (2.245e+010 bytes) *
Memory used by MATLAB: 5612 MB (5.885e+009 bytes)
Physical Memory (RAM): 32765 MB (3.436e+010 bytes)

Physical Memory (RAM):
In Use: 13611 MB ( 352b90000)
Free: 19153 MB ( 4ad1cb000)
Total: 32765 MB ( 7ffd5b000)
Page File (Swap space):
In Use: 13766 MB ( 35c67d000)
Free: 21404 MB ( 539ca0000)
Total: 35171 MB ( 89631d000)
Virtual Memory (Address Space):
In Use: 8344 MB ( 209860000)
Free: 8380263 MB (7fdf6780000)
Total: 8388607 MB (7fffffe0000)
Largest Contiguous Free Blocks:
1. [at 4d0883000] 8366183 MB (7fa867bd000)
2. [at 3192e2000] 7028 MB ( 1b741e000)
3. [at 7ff7ffe5000] 2046 MB ( 7fe2b000)
4. [at 203df0000] 1593 MB ( 639a0000)
5. [at 198ed2000] 674 MB ( 2a2ce000)
6. [at 1c61f2000] 475 MB ( 1dbfe000)
7. [at 105690000] 375 MB ( 177e0000)
8. [at f5c91000] 206 MB ( cedf000)
9. [at 7ff58b4c000] 190 MB ( bea4000)
10. [at 1879e2000] 170 MB ( aa8e000)
======= ==========
8378946 MB (7fda4263000)
This week I tired to save another structure, the size is about 400Mb, it gave me error again: ??? Error using ==> save Java exception occurred: java.lang.OutOfMemoryError: Java heap space My workstation configuration: wins xp 64bit 32G RAM memory Maximum possible array: 21408 MB (2.245e+010 bytes) * Memory available for all arrays: 21408 MB (2.245e+010 bytes) * Memory used by MATLAB: 5612 MB (5.885e+009 bytes) Physical Memory (RAM): 32765 MB (3.436e+010 bytes) Physical Memory (RAM): In Use: 13611 MB ( 352b90000) Free: 19153 MB ( 4ad1cb000) Total: 32765 MB ( 7ffd5b000) Page File (Swap space): In Use: 13766 MB ( 35c67d000) Free: 21404 MB ( 539ca0000) Total: 35171 MB ( 89631d000) Virtual Memory (Address Space): In Use: 8344 MB ( 209860000) Free: 8380263 MB (7fdf6780000) Total: 8388607 MB (7fffffe0000) Largest Contiguous Free Blocks: 1. [at 4d0883000] 8366183 MB (7fa867bd000) 2. [at 3192e2000] 7028 MB ( 1b741e000) 3. [at 7ff7ffe5000] 2046 MB ( 7fe2b000) 4. [at 203df0000] 1593 MB ( 639a0000) 5. [at 198ed2000] 674 MB ( 2a2ce000) 6. [at 1c61f2000] 475 MB ( 1dbfe000) 7. [at 105690000] 375 MB ( 177e0000) 8. [at f5c91000] 206 MB ( cedf000) 9. [at 7ff58b4c000] 190 MB ( bea4000) 10. [at 1879e2000] 170 MB ( aa8e000) ======= ========== 8378946 MB (7fda4263000)

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago Nov 16, 2011, 12:22 p.m. EST
I found one of the structures might be where the problem is:

For example,

Structure_test
ans =

u: [3974351x60 double] (143204734 stored)
ut: [3974351x60 double] (0 stored)
reacf: [3974351x60 double] (115800 stored)
tlist: [1x60 double]
This structure has four field variables, u, ut, reacf, tlist. It appears to me that Matlab can't handle 2^31 sized elements or more than 2gb file size. Is it true?

I think if we can find a way to store structure_test.ut, this problem will be solved.
I found one of the structures might be where the problem is: For example, Structure_test ans = u: [3974351x60 double] (143204734 stored) ut: [3974351x60 double] (0 stored) reacf: [3974351x60 double] (115800 stored) tlist: [1x60 double] This structure has four field variables, u, ut, reacf, tlist. It appears to me that Matlab can't handle 2^31 sized elements or more than 2gb file size. Is it true? I think if we can find a way to store structure_test.ut, this problem will be solved.

Ivar KJELBERG COMSOL Multiphysics(r) fan, retired, former "Senior Expert" at CSEM SA (CH)

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago Nov 16, 2011, 2:59 p.m. EST
Hi

2Gb is typically 32 bit adress space (in theory 4Gb) looks like your are not fully 64 bit adressing compatible there. But with 3.5 I'm not that astonished, you probably need to upgrade to latest version on matlab AND COMSOL and perhaps PC for such large model ?

And even with a 64 bit OS, if you format a disk to FAT32, you have a file size limit between 2-4 Gb. Do not use that for your scratch disk
--
Good luck
Ivar
Hi 2Gb is typically 32 bit adress space (in theory 4Gb) looks like your are not fully 64 bit adressing compatible there. But with 3.5 I'm not that astonished, you probably need to upgrade to latest version on matlab AND COMSOL and perhaps PC for such large model ? And even with a 64 bit OS, if you format a disk to FAT32, you have a file size limit between 2-4 Gb. Do not use that for your scratch disk -- Good luck Ivar

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago Nov 16, 2011, 3:27 p.m. EST
Thanks for your comments.
I am running a simulation using comsol 4.2 with matlab now to see if there is any changes.
However, I am still surprised that I could not save a 400M structure_test structure as below.
I have successfully saved a 1.5G structure (not exactly the same, but similar) before.

Too many variable in structure_test.u or structure_test.ut? But the total size is less than 1G.
Structure_test
ans =

u: [3974351x60 double] (143204734 stored)
ut: [3974351x60 double] (0 stored)

tks


Hi

2Gb is typically 32 bit adress space (in theory 4Gb) looks like your are not fully 64 bit adressing compatible there. But with 3.5 I'm not that astonished, you probably need to upgrade to latest version on matlab AND COMSOL and perhaps PC for such large model ?

And even with a 64 bit OS, if you format a disk to FAT32, you have a file size limit between 2-4 Gb. Do not use that for your scratch disk
--
Good luck
Ivar


Thanks for your comments. I am running a simulation using comsol 4.2 with matlab now to see if there is any changes. However, I am still surprised that I could not save a 400M structure_test structure as below. I have successfully saved a 1.5G structure (not exactly the same, but similar) before. Too many variable in structure_test.u or structure_test.ut? But the total size is less than 1G. Structure_test ans = u: [3974351x60 double] (143204734 stored) ut: [3974351x60 double] (0 stored) tks [QUOTE] Hi 2Gb is typically 32 bit adress space (in theory 4Gb) looks like your are not fully 64 bit adressing compatible there. But with 3.5 I'm not that astonished, you probably need to upgrade to latest version on matlab AND COMSOL and perhaps PC for such large model ? And even with a 64 bit OS, if you format a disk to FAT32, you have a file size limit between 2-4 Gb. Do not use that for your scratch disk -- Good luck Ivar [/QUOTE]

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago Dec 11, 2011, 6:19 p.m. EST
Hi,
I have the same problem with saving file using save('file_name.mat') which is bigger than 300 MB. I tried to edit java.opts using the instruction on MatLab web page:

www.mathworks.se/support/solutions/en/data/1-18I2C/index.html

or Comsol support (LiveLink) on:

www.comsol.fr/support/knowledgebase/1092/

but both do not work. The most strange thing is that the computer has 132 GB RAM with 32 cores and 1 TB harddrive but Java still gives this OutOfMemoryError.
Really do not know what can I do!


I am working with Comsol 3.5a with Matlab R2010a on OpenSuse 11.3
Hi, I have the same problem with saving file using save('file_name.mat') which is bigger than 300 MB. I tried to edit java.opts using the instruction on MatLab web page: http://www.mathworks.se/support/solutions/en/data/1-18I2C/index.html or Comsol support (LiveLink) on: http://www.comsol.fr/support/knowledgebase/1092/ but both do not work. The most strange thing is that the computer has 132 GB RAM with 32 cores and 1 TB harddrive but Java still gives this OutOfMemoryError. Really do not know what can I do! I am working with Comsol 3.5a with Matlab R2010a on OpenSuse 11.3

Ivar KJELBERG COMSOL Multiphysics(r) fan, retired, former "Senior Expert" at CSEM SA (CH)

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago Dec 12, 2011, 1:36 a.m. EST
Hi

probably it would work better in V4 where multi-core large (64bit) ram options have been further enhanced

--
Good luck
Ivar
Hi probably it would work better in V4 where multi-core large (64bit) ram options have been further enhanced -- Good luck Ivar

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago Dec 12, 2011, 8:59 a.m. EST
Hi,
thank you for quick answer.

But now I am confused what did you mean with the V4. Is it the version of OpenSuse (11.4) or I should use save command with '-v4' or is it some version of Matlab?
Also I changed the default option in File/Preferences/General/MAT-files to -v7.3 from -v7 so now it will maybe work better, I hope.

Thanks

p.
Hi, thank you for quick answer. But now I am confused what did you mean with the V4. Is it the version of OpenSuse (11.4) or I should use save command with '-v4' or is it some version of Matlab? Also I changed the default option in File/Preferences/General/MAT-files to -v7.3 from -v7 so now it will maybe work better, I hope. Thanks p.

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago Dec 12, 2011, 10:27 a.m. EST
Hi,

You might try to install comsol 4.2 and use comsol 4.2 with matlab (comsol 3.5 matlab compatibility)
Then use save -v7.3 xx command to save
Let me know if this works for u
Hi, You might try to install comsol 4.2 and use comsol 4.2 with matlab (comsol 3.5 matlab compatibility) Then use save -v7.3 xx command to save Let me know if this works for u

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago Dec 12, 2011, 12:49 p.m. EST
Hi,
well, the Comsol 4.2 is installed on the PC but the script I am using is written in Matlab with commands from Comsol 3.5 and I have heard that strating with Comsol 4.0 the commands was changed very hard. This is the reason I am still using Comsol 3.5 Is there anythig which can help me with the translation of 3.5 scripts to 4.?
And I suppose that V4 in message from Ivar is then a version of Comsol, isn't it?

p.
Hi, well, the Comsol 4.2 is installed on the PC but the script I am using is written in Matlab with commands from Comsol 3.5 and I have heard that strating with Comsol 4.0 the commands was changed very hard. This is the reason I am still using Comsol 3.5 Is there anythig which can help me with the translation of 3.5 scripts to 4.? And I suppose that V4 in message from Ivar is then a version of Comsol, isn't it? p.

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago Dec 12, 2011, 5:27 p.m. EST
You do not need to convert your code, there is a 'comsol 3.5 matlab compatibility' option in comsol 4.2.
You do not need to convert your code, there is a 'comsol 3.5 matlab compatibility' option in comsol 4.2.

Note that while COMSOL employees may participate in the discussion forum, COMSOL® software users who are on-subscription should submit their questions via the Support Center for a more comprehensive response from the Technical Support team.