Function quantize_fp16

src/quantization.cpp:68–76

This function is to convert any instructions specified in the input from double or float to float16 by inserting a convert operator. For the conversion, there could be cases of overflowing or underflowing, but it is uncommon. Run optimize_module() before converting to fp16 to const eval and fold in FP32 to avoid loss of precision.

// is uncommon. Run optimize_module() before converting to fp16 to const eval and fold in FP32 to
// avoid loss of precision.
void quantize_fp16(program& prog, const std::vector<std::string>& ins_names)
{
    run_passes(prog,
               {normalize_ops{},
                optimize_module{{"quantizelinear", "dequantizelinear"}},
                truncate_float_pass{ins_names, shape::half_type},
                optimize_module{{"quantizelinear", "dequantizelinear"}}},
               quant_tracer());
}

void quantize_bf16(program& prog, const std::vector<std::string>& ins_names)
{