Set some ENV for these DeepRec's features enabled by ENV. More Detail information is shown in https://deeprec.readthedocs.io/zh/latest/index.html. START_STATISTIC_STEP & STOP_STATISTIC_STEP: On CPU platform, DeepRec supports memory optimization in both stand-alone and distribute
()
| 916 | sys.exit() |
| 917 | |
| 918 | def set_env_for_DeepRec(): |
| 919 | ''' |
| 920 | Set some ENV for these DeepRec's features enabled by ENV. |
| 921 | More Detail information is shown in https://deeprec.readthedocs.io/zh/latest/index.html. |
| 922 | START_STATISTIC_STEP & STOP_STATISTIC_STEP: On CPU platform, DeepRec supports memory optimization |
| 923 | in both stand-alone and distributed trainging. It's default to open, and the |
| 924 | default start and stop steps of collection is 1000 and 1100. Reduce the initial |
| 925 | cold start time by the following settings. |
| 926 | MALLOC_CONF: On CPU platform, DeepRec can use memory optimization with the jemalloc library. |
| 927 | Please preload libjemalloc.so by `LD_PRELOAD=./libjemalloc.so.2 python ...` |
| 928 | DNNL_MAX_CPU_ISA: Specify the highest instruction set used by oneDNN (when the version is less than 2.5.0), |
| 929 | it will be set to AVX512_CORE_AMX to enable Intel CPU's feature. |
| 930 | ''' |
| 931 | os.environ['START_STATISTIC_STEP'] = '100' |
| 932 | os.environ['STOP_STATISTIC_STEP'] = '110' |
| 933 | os.environ['MALLOC_CONF'] = \ |
| 934 | 'background_thread:true,metadata_thp:auto,dirty_decay_ms:20000,muzzy_decay_ms:20000' |
| 935 | |
| 936 | def check_stock_tf(): |
| 937 | import pkg_resources |