Calibre PERC MTFlex Memory Prediction

The Memory Prediction feature is a machine learning algorithm that analyzes some aspects of design data to predict memory usage for certain portions of the run. These predicted memory values are then used to set the relevant memory suggestion value in Calibre MTflex. If CALIBRE_PERC_MTFLEX_MEMORY_SUGGESTION is set to a positive value, it takes precedence over the memory prediction values.

Currently, memory prediction is enabled by default in LDL P2P for the PEX and simulation modules only. Memory prediction in CD is more limited and only works with full path nets and must be enabled with the environment variable CALIBRE_PERC_MTFLEX_SIMULATION_MEMORY_PREDICT=1. Otherwise, an 800 GB default suggestion is used for PEX and resistance simulation in CD.

For the following discussion, a "slot" is a group of CPUs on a remote host that can run a remote job and is determined by the memory suggestion. For example, if the memory suggestion is 800 GB and there are three hosts with 400 GB, 800 GB, and 1600 GB of memory, respectively, the 400 GB host would get one slot (as each machine will get at least a single slot even if they are below the memory suggestion size), the 800 GB host would also get a single slot, and the 1600 GB host would get two slots because its memory is greater than or equal to twice the memory suggestion. The slot size is the same for all remote hosts and is determined by the smallest CPU count from the set of remote hosts.

When running Calibre MTflex, each separate remote host is allocated a number of slots based on the memory suggestion and the number of CPUs available. In the case where remote hosts have different numbers of CPUs but the same number of slots as determined by the available memory, CPU usage may be lower than expected. To illustrate, suppose host A has 16 CPUs and 200 GB of memory, host B has 32 CPUs and 200 GB of memory, and the memory suggestion is 200 GB. Each machine is assigned one slot (e.g., int(200/200) = 1, where "int" is a Tcl function). In this case, the CPU count assigned to both slots is 16 as determined by the smallest CPU count available. Host B only uses 16 CPUs. By way of contrast, suppose that host B has 500 GB of memory. Host A gets one slot as before while host B now gets two (int(500/200) = 2). Again, the CPU count assigned to every slot is 16 CPUs, but host B has two slots, so it uses all 32 CPUs.

Example:
This example assumes remote connections to three hosts where hardware memory capacity and runtime memory requirements are already known. Two of the remote hosts have 2 TB of memory, and the remaining one has 3 TB.
This is set for manual memory allocation:
# 2 TB limit for PEX, 1.5 TB limit for simulation
CALIBRE_PERC_MTFLEX_MEMORY_SUGGESTION="2000000 1500000"
The Calibre MTflex configuration file remote.config is as follows:
LAUNCH AUTOMATIC
  REMOTE HOST 3tb 32
  REMOTE HOST 2tb 32
  REMOTE HOST 2tb 32
This is the command that starts the job:
calibre -hier -perc -ldl -turbo -hyper -ys_hyper -remotefile remote.config rules
All of the remotes use 32 CPUs for operations that can run Calibre MTflex. All three remotes run one parasitic extraction process each (3 × 1) due to the 2 TB memory suggestion limit. The 2 TB remotes run one simulation process each, and he remaining remote runs two simulation processes (2 × 1, 1 × 2) due to the 1.5 TB memory suggestion limit.