MCPcopy
hub / github.com/ymcui/Chinese-LLaMA-Alpaca-2 / inference_hf.py

File inference_hf.py

scripts/inference/inference_hf.py:None–None  ·  view source on GitHub ↗

Source from the content-addressed store, hash-verified

1import argparse
2import json, os
3
4DEFAULT_SYSTEM_PROMPT = """You are a helpful assistant. 你是一个乐于助人的助手。"""

Callers

nothing calls this directly

Calls 9

replace_llama_attn_with_flash_attnFunction · 0.90
apply_attention_patchFunction · 0.90
apply_ntk_scaling_patchFunction · 0.90
speculative_sampleFunction · 0.90
generate_promptFunction · 0.70
from_pretrainedMethod · 0.45
evalMethod · 0.45
generateMethod · 0.45
save_pretrainedMethod · 0.45

Tested by

no test coverage detected