Functions — github.com/MrSuiChuan/data-warehouse-learning

hub / github.com/MrSuiChuan/data-warehouse-learning / functions

Functions532 in github.com/MrSuiChuan/data-warehouse-learning

⨍ Functions 532◇ Types & classes 165

↓ 1 callersMethodmap

(String value)

src/main/java/org/bigdatatechcir/learn_flink/part8_flink_checkpoint/TwoPhaseCommitSinkFunctionDemo.java:118

↓ 1 callersMethodmap

(String value)

src/main/java/org/bigdatatechcir/learn_flink/part8_flink_checkpoint/CheckpointModeDemo.java:108

↓ 1 callersMethodmap

(String value)

src/main/java/org/bigdatatechcir/learn_flink/part8_flink_checkpoint/GenericWriteAheadSinkDemo.java:113

↓ 1 callersFunctionprecompute_freqs_cis

(dim: int, end: int, theta: float = 10000.0)

src/main/java/org/bigdatatechcir/learn_llm/model/k_model.py:68

↓ 1 callersFunctionpreprocess

(sources, tokenizer, max_len, system_message: str = "You are a helpful assistant.")

src/main/java/org/bigdatatechcir/learn_llm/llm/finetune.py:87

↓ 1 callersMethodpretrain_sample

根据给定的起始文本生成样本。 :param start: 生成文本的起始提示词 :param num_samples: 要生成的文本样本数 :param max_new_tokens: 每个样本生成的最大 token

src/main/java/org/bigdatatechcir/learn_llm/model/model_sample.py:93

↓ 1 callersMethodprocess

(ProcessAllWindowFunction<Tuple3<String, Integer, Long>, Tuple2<String, Integer>, GlobalWindow>.Context contex

src/main/java/org/bigdatatechcir/learn_dinky/GlobalWindowDemo.java:77

↓ 1 callersMethodprocess

(String key, Context context,

src/main/java/org/bigdatatechcir/learn_flink/part6_flink_state/KeyedStateDemo.java:119

↓ 1 callersMethodprocess

(ProcessAllWindowFunction<Tuple3<String, Integer, Long>, Tuple2<String, Integer>, GlobalWindow>.Context contex

src/main/java/org/bigdatatechcir/learn_flink/part4_flink_window/GlobalWindowDemo.java:77

↓ 1 callersMethodprocess

(String s, ProcessWindowFunction<Tuple3<String, Integer, Long>, Tuple2<String, Integer>, String, TimeWindow>.C

src/main/java/org/bigdatatechcir/learn_flink/part4_flink_window/TumblingWindowDemo.java:82

↓ 1 callersMethodprocess

(String s, Context context, Iterable<Tuple3<String, Integer, Long>> elements, Collector<Tuple2<String, Integer

src/main/java/org/bigdatatechcir/learn_flink/part4_flink_window/SlidingWindowDemo.java:82

↓ 1 callersMethodprocess

(String s, Context context, Iterable<Tuple3<String, Integer, Long>> iterable, Collector<Tuple2<String, Integer

src/main/java/org/bigdatatechcir/learn_flink/part4_flink_window/SessionWindowDemo.java:83

↓ 1 callersMethodprocess

(String s, Context context, Iterable<Tuple3<String, Integer, Long>> elements, Collector<Tuple2<String, Integer

src/main/java/org/bigdatatechcir/learn_flink/part5_flink_watermark/WithIdLenessDemo.java:89

↓ 1 callersMethodprocess

(String s, Context context, Iterable<Tuple3<String, Integer, Long>> elements, Collector<Tuple2<String, Integer

src/main/java/org/bigdatatechcir/learn_flink/part5_flink_watermark/SideOutputDemo.java:110

↓ 1 callersMethodprocess

(String key, Context context, Iterable<Tuple3<String, Integer, Long>> elements, Collector<Tuple2<String, Integ

src/main/java/org/bigdatatechcir/learn_flink/part5_flink_watermark/PunctuatedWatermarkDemo.java:92

↓ 1 callersMethodprocess

(String s, Context context, Iterable<Tuple3<String, Integer, Long>> elements, Collector<Tuple2<String, Integer

src/main/java/org/bigdatatechcir/learn_flink/part5_flink_watermark/AllowedLatenessDemo.java:103

↓ 1 callersMethodprocess

(String s, Context context, Iterable<Tuple3<String, Integer, Long>> elements, Collector<Tuple2<String, Integer

src/main/java/org/bigdatatechcir/learn_flink/part5_flink_watermark/PeriodicWatermarkDemo.java:105

↓ 1 callersMethodprocess

(String s, Context context, Iterable<Event> elements, Collector<Tuple3<String, Integer, Long>> out)

src/main/java/org/bigdatatechcir/learn_flink/part3_flink_time/IngestionTimeDemo.java:41

↓ 1 callersMethodprocess

(String s, ProcessWindowFunction<Event, Tuple2<String, Integer>, String, TimeWindow>.Context context, Iterable

src/main/java/org/bigdatatechcir/learn_flink/part3_flink_time/EventTimeDemo.java:38

↓ 1 callersMethodprocess

(String s, Context context, Iterable<Event> elements, Collector<Tuple3<String, Integer, Long>> out)

src/main/java/org/bigdatatechcir/learn_flink/part3_flink_time/ProcessTimeDemo.java:31

↓ 1 callersMethodprocess

(String s, Context context, Iterable<Tuple3<String, Integer, Long>> elements, Collector<Tuple2<String, Integer

src/main/java/org/bigdatatechcir/learn_flink/part10_flink_restart/FailureRateRestartStrategyDemo.java:144

↓ 1 callersMethodprocess

(String s, Context context, Iterable<Tuple3<String, Integer, Long>> elements, Collector<Tuple2<String, Integer

src/main/java/org/bigdatatechcir/learn_flink/part10_flink_restart/FixedDelayRestartStrategyDemo.java:144

↓ 1 callersMethodprocess

(String s, Context context, Iterable<Tuple3<String, Integer, Long>> elements, Collector<Tuple2<String, Integer

src/main/java/org/bigdatatechcir/learn_flink/part7_flink_statebackend/RocksDBStateBackendDemo.java:137

↓ 1 callersMethodprocess

(String s, Context context, Iterable<Tuple3<String, Integer, Long>> elements, Collector<Tuple2<String, Integer

src/main/java/org/bigdatatechcir/learn_flink/part7_flink_statebackend/FsStateBackendDemo.java:133

↓ 1 callersMethodprocess

(String s, Context context, Iterable<Tuple3<String, Integer, Long>> elements, Collector<Tuple2<String, Integer

src/main/java/org/bigdatatechcir/learn_flink/part7_flink_statebackend/MemoryStateBackendDemo.java:136

↓ 1 callersMethodprocess

(String s, Context context, Iterable<Tuple3<String, Integer, Long>> elements, Collector<Tuple2<String, Integer

src/main/java/org/bigdatatechcir/learn_flink/part8_flink_checkpoint/TwoPhaseCommitSinkFunctionDemo.java:145

↓ 1 callersMethodprocess

(String s, Context context, Iterable<Tuple3<String, Integer, Long>> elements, Collector<Tuple2<String, Integer

src/main/java/org/bigdatatechcir/learn_flink/part8_flink_checkpoint/CheckpointModeDemo.java:135

↓ 1 callersMethodprocess

(String s, Context context, Iterable<Tuple3<String, Integer, Long>> elements, Collector<Tuple2<String, Integer

src/main/java/org/bigdatatechcir/learn_flink/part8_flink_checkpoint/GenericWriteAheadSinkDemo.java:140

↓ 1 callersMethodprocessNestedJson

处理嵌套的JSON字符串，避免转义

src/main/java/org/bigdatatechcir/warehouse/datageneration/userlog_code/UserLogApplication.java:105

↓ 1 callersMethodread_file_content

(cls, file_path: str)

src/main/java/org/bigdatatechcir/learn_llm/rag/utils.py:109

↓ 1 callersMethodread_markdown

(cls, file_path: str)

src/main/java/org/bigdatatechcir/learn_llm/rag/utils.py:131

↓ 1 callersMethodread_pdf

(cls, file_path: str)

src/main/java/org/bigdatatechcir/learn_llm/rag/utils.py:121

↓ 1 callersMethodread_text

(cls, file_path: str)

src/main/java/org/bigdatatechcir/learn_llm/rag/utils.py:144

↓ 1 callersFunctionread_texts_from_jsonl

读取JSONL文件并安全提取文本数据

src/main/java/org/bigdatatechcir/learn_llm/model/train_tokenizer.py:17

↓ 1 callersMethodrollback

()

src/main/java/org/bigdatatechcir/learn_flink/part8_flink_checkpoint/TwoPhaseCommitSinkFunctionDemo.java:271

↓ 1 callersMethodrun

(String... args)

src/main/java/org/bigdatatechcir/warehouse/datageneration/business_code/BusinessApplication.java:57

↓ 1 callersMethodrun

(String... args)

src/main/java/org/bigdatatechcir/warehouse/datageneration/userlog_code/UserLogApplication.java:43

↓ 1 callersMethodsft_sample

根据给定的起始文本生成样本。 :param start: 生成文本的起始提示词 :param num_samples: 要生成的文本样本数 :param max_new_tokens: 每个样本生成的最大 token

src/main/java/org/bigdatatechcir/learn_llm/model/model_sample.py:63

↓ 1 callersMethodsplitStatements

(Path path)

src/main/java/org/bigdatatechcir/warehouse/flink/tools/SqlSyntaxValidationRunner.java:52

↓ 1 callersFunctionsplit_text

将文本按指定长度切分成块

src/main/java/org/bigdatatechcir/learn_llm/model/download.py:20

↓ 1 callersMethodstore

(YRcanbusStore yrcanbusStore)

src/main/java/org/bigdatatechcir/learn_flink/part8_flink_checkpoint/TwoPhaseCommitSinkFunctionDemo.java:247

↓ 1 callersMethodtestConnection

()

src/main/java/org/bigdatatechcir/warehouse/datageneration/business_code/util/DbUtil.java:73

↓ 1 callersFunctiontrain_epoch

(epoch)

src/main/java/org/bigdatatechcir/learn_llm/model/ddp_sft_full.py:42

↓ 1 callersFunctiontrain_epoch

(epoch)

src/main/java/org/bigdatatechcir/learn_llm/model/ddp_pretrain.py:42

↓ 1 callersFunctiontrain_tokenizer

训练并保存自定义tokenizer

src/main/java/org/bigdatatechcir/learn_llm/model/train_tokenizer.py:77

MethodEvent

(String id, long timestamp, int value)

src/main/java/org/bigdatatechcir/learn_flink/part1_flink_apis/ProcessFunctionDemo.java:41

MethodEvent

()

src/main/java/org/bigdatatechcir/learn_flink/part3_flink_time/IngestionTimeDemo.java:64

MethodEvent

()

src/main/java/org/bigdatatechcir/learn_flink/part3_flink_time/EventTimeDemo.java:60

MethodEvent

()

src/main/java/org/bigdatatechcir/learn_flink/part3_flink_time/ProcessTimeDemo.java:54

MethodGenericWriteAheadSinkFunction

(CheckpointCommitter committer, TypeSerializer<TwoPhaseCommitSinkFunctionDemo.YRcanbusStore> serializer, Strin

src/main/java/org/bigdatatechcir/learn_flink/part8_flink_checkpoint/GenericWriteAheadSinkDemo.java:200

MethodKafkaConsumerThread

(Properties props, String topic, int threadNumber)

src/main/java/org/bigdatatechcir/learn_kafka/part3_kafka_consumer/ThirdMultiConsumerThreadDemo.java:44

MethodKafkaConsumerThread

(Properties props, String topic)

src/main/java/org/bigdatatechcir/learn_kafka/part3_kafka_consumer/FirstMultiConsumerThreadDemo.java:39

MethodOrder

()

src/main/java/org/bigdatatechcir/learn_flink/part1_flink_apis/StreamSQLDemo.java:59

MethodRecordsHandler

(ConsumerRecords<String, String> records)

src/main/java/org/bigdatatechcir/learn_kafka/part3_kafka_consumer/ThirdMultiConsumerThreadDemo.java:75

MethodYRCanbusTwoPhaseCommitSinkFunction

()

src/main/java/org/bigdatatechcir/learn_flink/part8_flink_checkpoint/TwoPhaseCommitSinkFunctionDemo.java:205

MethodYRcanbusStore

(String platform, String type, String day, String hour)

src/main/java/org/bigdatatechcir/learn_flink/part8_flink_checkpoint/TwoPhaseCommitSinkFunctionDemo.java:282

Method__getitem__

(self, i)

src/main/java/org/bigdatatechcir/learn_llm/llm/finetune.py:172

Method__getitem__

(self, index: int)

src/main/java/org/bigdatatechcir/learn_llm/model/dataset.py:25

Method__getitem__

(self, index: int)

src/main/java/org/bigdatatechcir/learn_llm/model/dataset.py:92

Method__init__

(self, raw_data, tokenizer, max_len: int)

src/main/java/org/bigdatatechcir/learn_llm/llm/finetune.py:159

Method__init__

(self, data_path, tokenizer, max_length=512)

src/main/java/org/bigdatatechcir/learn_llm/model/dataset.py:13

Method__init__

( self, dim: int = 768, n_layers: int = 12, n_heads: int = 16,

src/main/java/org/bigdatatechcir/learn_llm/model/k_model.py:16

Method__init__

(self, dim: int, eps: float)

src/main/java/org/bigdatatechcir/learn_llm/model/k_model.py:45

Method__init__

(self, args: ModelConfig)

src/main/java/org/bigdatatechcir/learn_llm/model/k_model.py:138

Method__init__

(self, layer_id: int, args: ModelConfig)

src/main/java/org/bigdatatechcir/learn_llm/model/k_model.py:251

Method__init__

(self, args: ModelConfig = None)

src/main/java/org/bigdatatechcir/learn_llm/model/k_model.py:287

Method__init__

初始化 TextGenerator 类，加载模型、设置设备和分词器等。

src/main/java/org/bigdatatechcir/learn_llm/model/model_sample.py:10

Method__init__

(self, path: str, is_api: bool)

src/main/java/org/bigdatatechcir/learn_llm/rag/Embeddings.py:25

Method__init__

(self, path: str = '', is_api: bool = True)

src/main/java/org/bigdatatechcir/learn_llm/rag/Embeddings.py:48

Method__init__

(self, path: str = '', is_api: bool = True)

src/main/java/org/bigdatatechcir/learn_llm/rag/Embeddings.py:88

Method__init__

(self, path: str = '', is_api: bool = True)

src/main/java/org/bigdatatechcir/learn_llm/rag/Embeddings.py:105

Method__init__

(self, path: str)

src/main/java/org/bigdatatechcir/learn_llm/rag/utils.py:31

Method__init__

(self, path: str = '')

src/main/java/org/bigdatatechcir/learn_llm/rag/utils.py:154

Method__init__

(self, document: List[str] = [''])

src/main/java/org/bigdatatechcir/learn_llm/rag/VectorBase.py:20

Method__init__

(self, path: str = '', model: str = "gpt-3.5-turbo-1106")

src/main/java/org/bigdatatechcir/learn_llm/rag/LLM.py:44

Method__init__

(self, path: str = '')

src/main/java/org/bigdatatechcir/learn_llm/rag/LLM.py:63

Method__init__

(self, path: str = '', model: str = "qwen-turbo")

src/main/java/org/bigdatatechcir/learn_llm/rag/LLM.py:80

Method__init__

(self, path: str = '', model: str = "glm-4")

src/main/java/org/bigdatatechcir/learn_llm/rag/LLM.py:99

Method__init__

(self, args: ModelArgs, is_causal=False)

src/main/java/org/bigdatatechcir/learn_llm/transformer/transformer.py:23

Method__init__

(self, features, eps=1e-6)

src/main/java/org/bigdatatechcir/learn_llm/transformer/transformer.py:105

Method__init__

(self, args)

src/main/java/org/bigdatatechcir/learn_llm/transformer/transformer.py:139

Method__init__

(self, args)

src/main/java/org/bigdatatechcir/learn_llm/transformer/transformer.py:159

Method__init__

(self, args)

src/main/java/org/bigdatatechcir/learn_llm/transformer/transformer.py:173

Method__init__

(self, args)

src/main/java/org/bigdatatechcir/learn_llm/transformer/transformer.py:200

Method__init__

(self, args)

src/main/java/org/bigdatatechcir/learn_llm/transformer/transformer.py:215

Method__init__

(self, args)

src/main/java/org/bigdatatechcir/learn_llm/transformer/transformer.py:242

Method__init__

(self, client: OpenAI, model: str = "Qwen/Qwen2.5-32B-Instruct", tools: List=[], verbose : bool = True)

src/main/java/org/bigdatatechcir/learn_llm/agent/src/core.py:15

Method__len__

(self)

src/main/java/org/bigdatatechcir/learn_llm/llm/finetune.py:169

Method__len__

(self)

src/main/java/org/bigdatatechcir/learn_llm/model/dataset.py:22

Method__len__

(self)

src/main/java/org/bigdatatechcir/learn_llm/model/dataset.py:53

Method_init_weights

(self, module)

src/main/java/org/bigdatatechcir/learn_llm/model/k_model.py:329

Method_init_weights

(self, module)

src/main/java/org/bigdatatechcir/learn_llm/transformer/transformer.py:276

Methodabort

(YRCanbusTransaction yrCanbusTransaction)

src/main/java/org/bigdatatechcir/learn_flink/part8_flink_checkpoint/TwoPhaseCommitSinkFunctionDemo.java:233

Functionadd

计算两个浮点数的和。 :param a: 第一个浮点数。 :param b: 第二个浮点数。 :return: 两个浮点数的和。

src/main/java/org/bigdatatechcir/learn_llm/agent/src/tools.py:13

MethodbeginTransaction

()

src/main/java/org/bigdatatechcir/learn_flink/part8_flink_checkpoint/TwoPhaseCommitSinkFunctionDemo.java:214

Methodcancel

()

src/main/java/org/bigdatatechcir/learn_dinky/GlobalWindowDemo.java:107

Methodcancel

()

src/main/java/org/bigdatatechcir/learn_flink/part6_flink_state/KeyedStateDemo.java:85

Methodcancel

()

src/main/java/org/bigdatatechcir/learn_flink/part6_flink_state/OperatorStateDemo.java:63

Methodcancel

()

src/main/java/org/bigdatatechcir/learn_flink/part4_flink_window/GlobalWindowDemo.java:107

Methodcancel

()

src/main/java/org/bigdatatechcir/learn_flink/part4_flink_window/TumblingWindowDemo.java:58

← previous next →201–300 of 532, ranked by callers