Functions — github.com/decodingai-magazine/llm-twin-course

hub / github.com/decodingai-magazine/llm-twin-course / functions

Functions323 in github.com/decodingai-magazine/llm-twin-course

⨍ Functions 323◇ Types & classes 158↳ Endpoints 5

↓ 1 callersMethodcreate_template

(self)

src/bonus_superlinked_rag/llm/prompt_templates.py:43

↓ 1 callersMethodcreate_template

(self, keep_top_k: int)

src/bonus_superlinked_rag/llm/prompt_templates.py:60

↓ 1 callersMethodcreate_vector_collection

(self, collection_name: str)

src/core/db/qdrant.py:34

↓ 1 callersFunctiondelete_endpoint_and_config

Deletes an AWS SageMaker endpoint and its associated configuration. Args: endpoint_name (str): The name of the SageMaker endpoint to dele

src/inference_pipeline/aws/delete_sagemaker_endpoint.py:17

↓ 1 callersMethoddownload_dataset

(self, dataset_id: str, split: str = "train")

src/training_pipeline/finetune.py:33

↓ 1 callersMethoddownload_dataset

(self, dataset_id: str, split: str = "train")

src/training_pipeline/download_dataset.py:21

↓ 1 callersMethoddownload_file

(self, bucket_name: str | None, object_name: str, download_path: str)

src/bonus_superlinked_rag/server/runner/poller/app/resource_handler/resource_handler.py:40

↓ 1 callersMethodembedd

(self, data_model: DataModel)

src/feature_pipeline/data_logic/embedding_data_handlers.py:20

↓ 1 callersMethodensure_folder

(self)

src/bonus_superlinked_rag/server/runner/executor/app/service/file_handler_service.py:21

↓ 1 callersMethodextract

(self, link: str, **kwargs)

src/data_crawling/crawlers/base.py:14

↓ 1 callersFunctionextract_substrings

( text: str, min_length: int = 1000, max_length: int = 2000 )

src/feature_pipeline/generate_dataset/chunk_documents.py:12

↓ 1 callersMethodfetch_all_cleaned_content

(self, collection_name: str)

src/feature_pipeline/generate_dataset/generate.py:198

↓ 1 callersFunctionfinetune

( model_name: str, output_dir: str, dataset_id: str, max_seq_length: int = 2048, load_in_4

src/training_pipeline/finetune.py:130

↓ 1 callersMethodformat_batch

(cls, context_msg: str, data_points: list, start_index: int)

src/feature_pipeline/generate_dataset/generate.py:63

↓ 1 callersMethodformat_data

(cls, data_points: list, is_example: bool, start_index: int)

src/feature_pipeline/generate_dataset/generate.py:53

↓ 1 callersMethodformat_prompt

( self, system_prompt, prompt_template: PromptTemplate, prompt_template_variab

src/inference_pipeline/llm_twin.py:84

↓ 1 callersMethodformat_prompt

( cls, inference_posts: list, data_type: str, start_index: int )

src/feature_pipeline/generate_dataset/generate.py:70

↓ 1 callersMethodgenerate_training_data

( self, collection_name: str, data_type: str, batch_size: int = 3 )

src/feature_pipeline/generate_dataset/generate.py:93

↓ 1 callersMethodget_bucket

(self)

src/bonus_superlinked_rag/server/runner/poller/app/resource_handler/resource_handler.py:43

↓ 1 callersFunctionget_clean_collection

(data_type: str)

src/feature_pipeline/data_flow/stream_output.py:96

↓ 1 callersMethodget_collection

(self, collection_name: str)

src/core/db/qdrant.py:26

↓ 1 callersMethodget_crawler

(self, url: str)

src/data_crawling/dispatcher.py:17

↓ 1 callersMethodget_database

(self)

src/core/db/mongo.py:29

↓ 1 callersMethodget_registry

(app_module_path: str)

src/bonus_superlinked_rag/server/runner/executor/app/util/registry_loader.py:25

↓ 1 callersMethodget_resource_handler

(app_location: AppLocation)

src/bonus_superlinked_rag/server/runner/poller/app/resource_handler/resource_handler_factory.py:26

↓ 1 callersMethodget_system_prompt

(cls, data_type: str)

src/feature_pipeline/generate_dataset/generate.py:44

↓ 1 callersMethodget_task_status_by_name

(self, name: str)

src/bonus_superlinked_rag/server/runner/executor/app/service/data_loader.py:68

↓ 1 callersFunctionget_vector_collection

(data_type: str)

src/feature_pipeline/data_flow/stream_output.py:107

↓ 1 callersFunctionhandler

(event, context: LambdaContext | None = None)

src/data_crawling/main.py:18

↓ 1 callersFunctionhas_recency_space

(indices: Sequence[Index])

src/bonus_superlinked_rag/server/runner/executor/app/middleware/lifespan_event.py:92

↓ 1 callersFunctioninference

( model: Any, tokenizer: Any, prompt: str = "Write a paragraph to introduce supervised fine-tuning

src/training_pipeline/finetune.py:236

↓ 1 callersMethodinitialize_gcs_client

Initialize the GCS client, with fallback to credentials file if necessary.

src/bonus_superlinked_rag/server/runner/poller/app/resource_handler/gcs/gcs_resource_handler.py:29

↓ 1 callersMethodinitialize_s3_client

Initialize the S3 client, with fallback to credentials file if necessary.

src/bonus_superlinked_rag/server/runner/poller/app/resource_handler/s3/s3_resource_handler.py:36

↓ 1 callersFunctioninsert_data_to_mongodb

Insert data into a MongoDB collection. :param uri: MongoDB URI :param database_name: Name of the database :param collection_name: Na

src/data_cdc/test_cdc.py:4

↓ 1 callersFunctionload_model

( model_name: str, max_seq_length: int, load_in_4bit: bool, lora_rank: int, lora_alpha: in

src/training_pipeline/finetune.py:98

↓ 1 callersMethodlogin

(self)

src/data_crawling/crawlers/linkedin.py:144

↓ 1 callersFunctionmain

()

src/inference_pipeline/aws/deploy_sagemaker_endpoint.py:5

↓ 1 callersFunctionmain

()

src/inference_pipeline/evaluation/evaluate_rag.py:41

↓ 1 callersFunctionmain

()

src/inference_pipeline/evaluation/evaluate_monitoring.py:22

↓ 1 callersFunctionmain

()

src/inference_pipeline/evaluation/evaluate.py:31

↓ 1 callersFunctionmain

Parse arguments and execute the docker-compose command.

src/bonus_superlinked_rag/server/tools/deploy.py:133

↓ 1 callersMethodnotify_executor

Notify the executor API about the new version by making a PUT request to /reload.

src/bonus_superlinked_rag/server/runner/poller/app/resource_handler/resource_handler.py:82

↓ 1 callersMethodparse

Parse the application location URL to extract the storage type, bucket, and path.

src/bonus_superlinked_rag/server/runner/poller/app/app_location_parser/app_location_parser.py:52

↓ 1 callersMethodparse_app_location_config

Parse the configuration from the YAML file.

src/bonus_superlinked_rag/server/runner/poller/app/poller/poller.py:40

↓ 1 callersMethodpersist

(self)

src/bonus_superlinked_rag/server/runner/executor/app/service/persistence_service.py:36

↓ 1 callersMethodpush_to_comet

( self, train_test_split: tuple[list[dict], list[dict]], data_type: str, colle

src/feature_pipeline/generate_dataset/generate.py:155

↓ 1 callersMethodread_json

(self, filename: str)

src/feature_pipeline/generate_dataset/file_handler.py:7

↓ 1 callersMethodregister_data_loader_sources

(self, data_loader_sources: Sequence[DataLoaderSource])

src/bonus_superlinked_rag/server/runner/executor/app/service/data_loader.py:43

↓ 1 callersFunctionregister_dependencies

()

src/bonus_superlinked_rag/server/runner/executor/app/dependency_register.py:27

↓ 1 callersFunctionremove_emojis_and_symbols

(text)

src/feature_pipeline/utils/cleaning.py:68

↓ 1 callersFunctionremove_emojis_and_symbols

(text)

src/bonus_superlinked_rag/utils/cleaning.py:68

↓ 1 callersFunctionreplace_urls_with_placeholder

(text, placeholder="[URL]")

src/feature_pipeline/utils/cleaning.py:86

↓ 1 callersFunctionreplace_urls_with_placeholder

(text, placeholder="[URL]")

src/bonus_superlinked_rag/utils/cleaning.py:86

↓ 1 callersMethodrerank

(self, documents: list[Document], keep_top_k: int)

src/bonus_superlinked_rag/rag/retriever.py:94

↓ 1 callersMethodrestart

Restart the API via supervisor XML-RPC and return the result.

src/bonus_superlinked_rag/server/runner/executor/app/service/supervisor_service.py:22

↓ 1 callersMethodrestore

(self)

src/bonus_superlinked_rag/server/runner/executor/app/service/persistence_service.py:40

↓ 1 callersMethodretrieve_top_k

(self, k: int, to_expand_to_n_queries: int)

src/bonus_superlinked_rag/rag/retriever.py:60

↓ 1 callersFunctionrun_finetuning_on_sagemaker

( num_train_epochs: int = 3, per_device_train_batch_size: int = 2, learning_rate: float = 3e-4,

src/training_pipeline/run_on_sagemaker.py:23

↓ 1 callersFunctionsave_model

( model: Any, tokenizer: Any, output_dir: str, push_to_hub: bool = False, repo_id: Optiona

src/training_pipeline/finetune.py:252

↓ 1 callersMethodscroll

(self, collection_name: str, limit: int)

src/core/db/qdrant.py:64

↓ 1 callersMethodsend_prompt

(self, prompt: str)

src/feature_pipeline/generate_dataset/llm_communication.py:20

↓ 1 callersMethodset_extra_driver_options

(self, options: Options)

src/data_crawling/crawlers/base.py:42

↓ 1 callersFunctionsetup_application

(app: FastAPI)

src/bonus_superlinked_rag/server/runner/executor/app/middleware/lifespan_event.py:50

↓ 1 callersFunctionstream_process

()

src/data_cdc/cdc.py:13

↓ 1 callersFunctionteardown_application

(_: FastAPI)

src/bonus_superlinked_rag/server/runner/executor/app/middleware/lifespan_event.py:85

↓ 1 callersFunctiontruncate_text_to_max_tokens

Truncates text to not exceed max_tokens while trying to preserve complete sentences. Args: text: The text to truncate max_tokens:

src/inference_pipeline/utils.py:11

↓ 1 callersFunctionunbold_text

(text)

src/feature_pipeline/utils/cleaning.py:10

↓ 1 callersFunctionunbold_text

(text)

src/bonus_superlinked_rag/utils/cleaning.py:10

↓ 1 callersFunctionunitalic_text

(text)

src/feature_pipeline/utils/cleaning.py:49

↓ 1 callersFunctionunitalic_text

(text)

src/bonus_superlinked_rag/utils/cleaning.py:49

↓ 1 callersFunctionvalidate_config

(config_path: str)

src/bonus_superlinked_rag/server/tools/deploy.py:57

Method__call__

Possible changes to the value of the `__init__` argument do not affect the returned instance.

src/bonus_superlinked_rag/singleton.py:19

src/bonus_superlinked_rag/mq.py:43

Method__exit__

(self, exc_type, exc_val, exc_tb)

src/core/mq.py:44

Method__exit__

(self, exc_type, exc_val, exc_tb)

src/bonus_superlinked_rag/mq.py:47

Method__init__

(self, mock: bool = False)

src/inference_pipeline/llm_twin.py:19

Method__init__

( self, name: str = "style_metric", model_name: str = settings.OPENAI_MODEL_ID )

src/inference_pipeline/evaluation/style.py:23

Method__init__

( self, host: str | None = None, port: int | None = None, username: str | None

src/core/mq.py:22

Method__init__

(self, query: str)

src/core/rag/retriever.py:23

Method__init__

(self)

src/core/db/qdrant.py:13

Method__init__

(self, queue_name: str, resume_state: MessageT | None = None)

src/feature_pipeline/data_flow/stream_input.py:23

Method__init__

(self, connection: QdrantDatabaseConnector, sink_type: str)

src/feature_pipeline/data_flow/stream_output.py:16

Method__init__

(self, connection: QdrantDatabaseConnector)

src/feature_pipeline/data_flow/stream_output.py:57

Method__init__

(self, connection: QdrantDatabaseConnector)

src/feature_pipeline/data_flow/stream_output.py:77

Method__init__

( self, file_handler: FileHandler, api_communicator: GptCommunicator, data_for

src/feature_pipeline/generate_dataset/generate.py:83

Method__init__

(self, gpt_model: str = settings.OPENAI_MODEL_ID)

src/feature_pipeline/generate_dataset/llm_communication.py:16

Method__init__

(self)

src/data_crawling/dispatcher.py:11

Method__init__

(self)

src/data_crawling/crawlers/custom_article.py:16

Method__init__

(self, scroll_limit: int = 5)

src/data_crawling/crawlers/base.py:18

Method__init__

(self, ignore=(".git", ".toml", ".lock", ".png"))

src/data_crawling/crawlers/github.py:17

Method__init__

(self, base_url=settings.SUPERLINKED_SERVER_URL)

src/bonus_superlinked_rag/superlinked_client.py:16

Method__init__

( self, host: str | None = None, port: int | None = None, username: str | None

src/bonus_superlinked_rag/mq.py:25

Method__init__

(self, queue_name: str, resume_state: MessageT | None = None)

src/bonus_superlinked_rag/data_flow/stream_input.py:33

Method__init__

(self, client: SuperlinkedClient)

src/bonus_superlinked_rag/data_flow/stream_output.py:11

Method__init__

(self, client: SuperlinkedClient)

src/bonus_superlinked_rag/data_flow/stream_output.py:19

Method__init__

(self, file_handler_service: FileHandlerService)

src/bonus_superlinked_rag/server/runner/executor/app/service/file_object_serializer.py:30

Method__init__

(self, server_proxy: ServerProxy)

src/bonus_superlinked_rag/server/runner/executor/app/service/supervisor_service.py:19

Method__init__

(self, app_config: AppConfig, hash_type: HashType | None = None)

src/bonus_superlinked_rag/server/runner/executor/app/service/file_handler_service.py:13

Method__init__

(self, app_config: AppConfig)

src/bonus_superlinked_rag/server/runner/executor/app/service/data_loader.py:38

← previous next →101–200 of 323, ranked by callers