You can change the java maximum heap size (-Xmx) in your /bin/mduml.properties* file, in the line:
JAVA_ARGS=-Xmx800M
The maximum theoretical heap limit for the 32-bit JVM is 4G. Due to various additional constraints such as available swap, kernel address space usage, memory fragmentation, and VM overhead, in practice the limit can be much lower. On most modern 32-bit Windows systems the maximum heap size will range from 1.4G to 1.6G. On 32-bit Solaris kernels the address space is limited to 2G. On 64-bit operating systems running the 32-bit VM, the max heap size can be higher, approaching 4G on many Solaris systems.
On 64-bit VMs, you have 64 bits of addressability to work with resulting in a maximum Java heap size limited only by the amount of physical memory and swap space your system provides.
More information about OutOfMemory problems you can find in readme.html by "OutOfMemory problem" paragraph. This file is located in <MagicDraw installation directory>.
* If you are using MagicDraw 17.0.5 or later, the name of the property file is magicdraw.properties.