Functions — github.com/Mooncake-Labs/moonlink

hub / github.com/Mooncake-Labs/moonlink / functions

Functions2,717 in github.com/Mooncake-Labs/moonlink

⨍ Functions 2,717◇ Types & classes 409

↓ 6 callersFunctioncreate_test_file_catalog

Test util to create file catalog.

src/moonlink/src/storage/table/iceberg/file_catalog_test_utils.rs:12

↓ 6 callersMethoddrop_table

Request to drop iceberg table and block wait its completion.

src/moonlink/src/table_handler/test_utils.rs:217

↓ 6 callersMethoddump_all_for_test

(&self)

src/moonlink/src/storage/cache/metadata/moka_cache_test.rs:13

↓ 6 callersFunctionexecute_test_upload

Test util function to send upload request.

src/moonlink_service/src/test_utils.rs:250

↓ 6 callersMethodget_file_indices

Get flushed file indices for the current streaming commit.

src/moonlink/src/storage/mooncake_table/transaction_stream.rs:91

↓ 6 callersFunctionget_index_block_files_size

Test util function to get overall file size for all index block files from the given mooncake table.

src/moonlink/src/storage/mooncake_table/table_accessor_test_utils.rs:226

↓ 6 callersMethodget_latest_wal_metadata

(&self)

src/moonlink/src/table_handler/test_utils.rs:461

↓ 6 callersMethodget_main_transaction_tracker

(&self)

src/moonlink/src/storage/wal.rs:373

↓ 6 callersFunctionget_new_compacted_local_file_size_and_id

Test util to get new compacted data file size and file id for the given mooncake table. Assert the file is of local filepath, and there's only one new

src/moonlink/src/storage/mooncake_table/table_accessor_test_utils.rs:251

↓ 6 callersMethodget_root_path

Get root path for the given filesystem config.

src/moonlink/src/storage/filesystem/storage_config.rs:118

↓ 6 callersFunctionget_table_id

(client: &tokio_postgres::Client, table_name: &str)

src/moonlink_connectors/src/pg_replicate/tests/pg_type_tests.rs:30

↓ 6 callersMethodhas_ongoing_flush

(&self)

src/moonlink/src/storage/mooncake_table.rs:1045

↓ 6 callersMethodis_deleted

(&self, row_idx: usize)

src/moonlink/src/storage/mooncake_table/delete_vector.rs:107

↓ 6 callersMethodload_metadata

Load metadata and its location foe the given table.

src/moonlink/src/storage/table/iceberg/file_catalog.rs:715

↓ 6 callersFunctionmake_req_with_metrics

Util functions to create otel service request.

src/moonlink_service/src/otel/test_utils.rs:49

↓ 6 callersFunctionmake_schema_via_field_schema

()

src/moonlink_connectors/src/rest_ingest/json_converter.rs:1241

↓ 6 callersFunctionmake_struct

Util function to make moonlink struct type.

src/moonlink_service/src/otel/otel_to_moonlink_pb.rs:113

↓ 6 callersMethodnamespace_exists

Check if namespace exists in catalog.

src/moonlink/src/storage/table/iceberg/file_catalog.rs:389

↓ 6 callersMethodperform_data_compaction

Perform data compaction, whose completion will be notified separately in async style.

src/moonlink/src/storage/mooncake_table.rs:1444

↓ 6 callersFunctionprepare_state_3

Prepare state 3 for read usage: remote, local, in use # Arguments read_then_snapshot: to reach state "remote" and "in use", we could either "persist

src/moonlink/src/storage/mooncake_table/data_file_state_tests.rs:250

↓ 6 callersFunctionprepare_test_deletion_vector_for_read

======================== Test util function for read ======================== Prepare persisted data files and their deletion vector in mooncake tabl

src/moonlink/src/storage/mooncake_table/deletion_vector_puffin_state_tests.rs:61

↓ 6 callersFunctionprepare_test_disk_file_for_read

Prepare persisted data files in mooncake table. Rows are committed and flushed with LSN 1.

src/moonlink/src/storage/mooncake_table/data_file_state_tests.rs:113

↓ 6 callersFunctionprepare_test_disk_files_for_compaction

There're two things different from use for read: - we can only check data compaction after its completion - data compaction only happens when there's

src/moonlink/src/storage/mooncake_table/data_file_state_tests.rs:1329

↓ 6 callersFunctionproto_to_moonlink_row

(p: moonlink_pb::MoonlinkRow)

src/moonlink/src/row/proto_converter.rs:18

↓ 6 callersFunctionread

(reader: &mut R)

src/moonlink_rpc/src/lib.rs:49

↓ 6 callersMethodread_object_as_string

(&self, object: &str)

src/moonlink/src/storage/filesystem/accessor/filesystem_accessor.rs:249

↓ 6 callersFunctionread_usize

(reader: &mut R)

src/moonlink_table_metadata/src/table_metadata.rs:182

↓ 6 callersFunctionrecreate_directory

Util function to delete and re-create the given directory.

src/moonlink_backend/src/file_utils.rs:62

↓ 6 callersMethodset_data_compaction_res

Set data compaction result, which will be sync-ed to mooncake and iceberg snapshot in the next periodic snapshot iteration.

src/moonlink/src/storage/mooncake_table.rs:772

↓ 6 callersMethodshutdown_connection

Gracefully shutdown a replication connection identified by its URI. If postgres drop all is false, then we will not drop the PostgreSQL publication an

src/moonlink_backend/src/lib.rs:431

↓ 6 callersMethodtable_exists

Check if a table exists in the catalog.

src/moonlink/src/storage/table/iceberg/file_catalog.rs:561

↓ 6 callersFunctiontest_unbuffered_stream_writer_impl

Util function to test stream writer.

src/moonlink/src/storage/filesystem/test_utils/writer_test_utils.rs:7

↓ 6 callersMethodupdate

Update statistics

src/moonlink/src/storage/table/iceberg/parquet_stats_utils.rs:366

↓ 6 callersFunctionvalidate_only_new_data_files_in_snapshot

Validate new snapshot with new data files created, but no deletion vector.

src/moonlink/src/storage/table/iceberg/state_tests.rs:337

↓ 5 callersMethodabort_in_stream_batch

(&mut self, xact_id: u32)

src/moonlink/src/storage/mooncake_table/transaction_stream.rs:334

↓ 5 callersMethodbegin_readonly_transaction

Starts a read-only transaction with repeatable read isolation level

src/moonlink_connectors/src/pg_replicate/clients/postgres.rs:106

↓ 5 callersFunctionconvert_avro_to_arrow_schema

Convert an Avro schema to an Arrow schema

src/moonlink_connectors/src/rest_ingest/avro_converter.rs:115

↓ 5 callersMethodcopy_from_remote_to_local

(&self, src: &str, dst: &str)

src/moonlink/src/storage/filesystem/accessor/filesystem_accessor.rs:406

↓ 5 callersFunctioncreate_file_index_2

( directory: std::path::PathBuf, data_file: MooncakeDataFileRef, start_file_id: u64, )

src/moonlink/src/storage/compaction/test_utils.rs:123

↓ 5 callersFunctioncreate_gcs_catalog

()

src/moonlink/src/storage/table/iceberg/file_catalog_test.rs:99

↓ 5 callersFunctioncreate_iceberg_snapshot

Test util function to perform an iceberg snapshot, block wait its completion and gets its result.

src/moonlink/src/storage/mooncake_table/table_operation_test_utils.rs:506

↓ 5 callersMethodcreate_iterator

( &'a self, metadata: &'a GlobalIndex, file_id_remap: &'a Vec<u32>, )

src/moonlink/src/storage/index/persisted_bucket_hash_map.rs:116

↓ 5 callersFunctioncreate_mooncake_table

( mooncake_table_metadata: Arc<MooncakeTableMetadata>, iceberg_table_config: IcebergTableConfig, o

src/moonlink/src/table_handler/profile_test.rs:429

↓ 5 callersFunctioncreate_s3_catalog

()

src/moonlink/src/storage/table/iceberg/file_catalog_test.rs:91

↓ 5 callersFunctioncreate_test_accessor

Create accessor with optional throttle configuration

src/moonlink/src/storage/filesystem/accessor/throttle_test.rs:12

↓ 5 callersFunctiondatabase_url

()

src/moonlink_connectors/src/pg_replicate/tests/test_utils.rs:17

↓ 5 callersFunctionexecute_test_ingest

Test util function to send ingest request.

src/moonlink_service/src/test_utils.rs:215

↓ 5 callersMethodfinalize

Finalize the current manifest file and return.

src/moonlink/src/storage/table/iceberg/data_file_manifest_manager.rs:93

↓ 5 callersMethodfinalize_current_batch

Finalize the current batch, adding it to filled_batches and preparing for a new batch

src/moonlink/src/storage/mooncake_table/data_batches.rs:171

↓ 5 callersMethodfinish

Finish the current batch and return a RecordBatch.

src/moonlink_connectors/src/pg_replicate/initial_copy_writer.rs:122

↓ 5 callersMethodforce_data_compaction_and_sync

Force a data compaction operation, and block wait its completion.

src/moonlink/src/table_handler/test_utils.rs:303

↓ 5 callersMethodget_all_table_metadata_entries

(&self)

src/moonlink_metadata_store/src/sqlite/sqlite_metadata_store.rs:31

↓ 5 callersMethodget_and_next

Increment the id by 1, and return the id before change. Relaxed ordering is used here because the counter is only used for internal state tracking, n

src/moonlink/src/storage/mooncake_table/batch_id_counter.rs:40

↓ 5 callersFunctionget_control_pid

Helper: fetch backend pid of the control-plane client via SELECT pg_backend_pid()

src/moonlink_backend/tests/test_control_plane_recovery.rs:15

↓ 5 callersMethodget_current_table_state

Get current table state.

src/moonlink/src/storage/mooncake_table/table_status_reader.rs:32

↓ 5 callersMethodget_flush_lsn

(&self)

src/moonlink/src/storage/table/iceberg/iceberg_snapshot_fetcher.rs:45

↓ 5 callersMethodget_flushed_data_files

Get flushed data files for the current streaming commit.

src/moonlink/src/storage/mooncake_table/transaction_stream.rs:87

↓ 5 callersFunctionget_lsn_for_pending_xact

DevNote: This is a trick to track xact of uncommitted deletions we set first 32 bits to 1, so it will be 'uncommitted' as the value is larger than any

src/moonlink/src/storage/mooncake_table/transaction_stream.rs:142

↓ 5 callersFunctionget_moonlink_row

Test util function to get the moonlink row of the request index.

src/moonlink/src/storage/table/iceberg/compaction_tests.rs:52

↓ 5 callersFunctionget_only_index_block_file_id

Test util to get all index block file ids for the table, and assert there's only one file.

src/moonlink/src/storage/mooncake_table/table_accessor_test_utils.rs:65

↓ 5 callersFunctionget_only_remote_data_file_id

Test util function to assert there's only one data file in table snapshot, and it indicates remote file.

src/moonlink/src/storage/mooncake_table/table_accessor_test_utils.rs:160

↓ 5 callersMethodget_persisted_table_lsn

Get the largest LSN where all updates have been persisted into iceberg. The difference between "persisted table LSN" and "iceberg snapshot LSN" is, su

src/moonlink/src/table_handler/table_handler_state.rs:253

↓ 5 callersMethodget_start_file

(&self)

src/moonlink/src/storage/wal.rs:226

↓ 5 callersMethodget_table_unique_file_id

Util function to get table unique file id.

src/moonlink/src/storage/mooncake_table/snapshot.rs:204

↓ 5 callersFunctionget_test_object_storage_cache

Test util function to create object storage cache, with local filesystem optimization disabled.

src/moonlink/src/storage/cache/object_storage/test_utils.rs:58

↓ 5 callersFunctionlist_tables

(client: &reqwest::Client)

src/moonlink_service/src/test_utils.rs:706

↓ 5 callersMethodlist_tables

List tables from namespace, return error if the given namespace doesn't exist.

src/moonlink/src/storage/table/iceberg/file_catalog.rs:424

↓ 5 callersFunctionload_blob_from_puffin_file

Load blob from the given puffin filepath. Note: this function assumes there's only one blob in the puffin file.

src/moonlink/src/storage/table/iceberg/puffin_utils.rs:45

↓ 5 callersFunctionmake_datetime_schema

()

src/moonlink_connectors/src/rest_ingest/json_converter.rs:290

↓ 5 callersMethodmerge_with

Merge with another batch deletion vector.

src/moonlink/src/storage/mooncake_table/delete_vector.rs:67

↓ 5 callersFunctionparse_timestamp_with_timezone

Parse an RFC3339/ISO8601 timestamp with optional schema timezone and normalize to UTC Follows the same behavior as pg -> moonlink: canonicalize to UTC

src/moonlink_connectors/src/rest_ingest/datetime_utils.rs:53

↓ 5 callersFunctionprepare_state_1

================================ States preparation functions ================================ Prepare state 1 for usage: remote, no local, not used

src/moonlink/src/storage/mooncake_table/data_file_state_tests.rs:179

↓ 5 callersFunctionprepare_state_2

Prepare state 2 for usage: remote, no local, in use

src/moonlink/src/storage/mooncake_table/data_file_state_tests.rs:209

↓ 5 callersMethodprocess_row_request_sync

Synchronous row processing

src/moonlink_connectors/src/rest_ingest/rest_source.rs:335

↓ 5 callersMethodremove_directory

(&self, directory: &str)

src/moonlink/src/storage/filesystem/accessor/filesystem_accessor.rs:182

↓ 5 callersFunctionreplay

(replay_filepath: &str)

src/moonlink/src/table_handler/chaos_replay.rs:190

↓ 5 callersMethodsend_event_request

(&self, request: EventRequest)

src/moonlink_backend/src/lib.rs:450

↓ 5 callersMethodserialize

Serialize the deletion vector into `Blob` to write to puffin files. Serialization storage format: | len for magic and vector | magic | vector | crc32

src/moonlink/src/storage/table/iceberg/deletion_vector.rs:87

↓ 5 callersMethodstats_object

(&self, object: &str)

src/moonlink/src/storage/filesystem/accessor/filesystem_accessor.rs:236

↓ 5 callersMethodstream_mut

(&mut self)

src/moonlink_datafusion/src/connection_pool.rs:110

↓ 5 callersFunctionsync_iceberg_snapshot

( receiver: &mut Receiver<TableEvent>, )

src/moonlink/src/storage/mooncake_table/table_operation_test_utils.rs:352

↓ 5 callersMethodtake_as_batch_delete_vector

Convert self to `BatchDeletionVector`, after which self ownership is terminated.

src/moonlink/src/storage/table/iceberg/deletion_vector.rs:241

↓ 5 callersMethodtake_as_table_commit

Take as [`TableCommit`].

src/moonlink/src/storage/table/iceberg/table_commit_proxy.rs:13

↓ 5 callersFunctiontest_get_hashes_for_index

(values: &[u64])

src/moonlink/src/storage/index/persisted_bucket_hash_map.rs:892

↓ 5 callersFunctiontest_iceberg_table_config

Test util function to get iceberg table config for testing purpose.

src/moonlink/src/storage/mooncake_table/test_utils.rs:51

↓ 5 callersFunctiontest_row_3

()

src/moonlink/src/storage/table/iceberg/tests.rs:148

↓ 5 callersFunctiontest_update_schema_impl

( catalog: &mut dyn MoonlinkCatalog, namespace: String, table_name: String, )

src/moonlink/src/storage/table/iceberg/catalog_test_impl.rs:275

↓ 5 callersFunctiontest_update_table_impl

( catalog: &mut dyn MoonlinkCatalog, namespace: String, table_name: String, )

src/moonlink/src/storage/table/iceberg/catalog_test_impl.rs:207

↓ 5 callersMethodtmp

(&self)

src/moonlink_backend/tests/common.rs:99

↓ 5 callersMethodupdate_table

Update a table to the catalog, which writes metadata file and version hint file.

src/moonlink/src/storage/table/iceberg/file_catalog.rs:590

↓ 4 callersMethodadd_table

( &self, src_table_name: String, src_table_id: SrcTableId, schema: Arc<Schema>

src/moonlink_connectors/src/rest_ingest.rs:100

↓ 4 callersMethodadd_table

Add a table to the REST sink # Arguments persist_lsn: only assigned at recovery, used to indicate and update commit LSN and replication LSN.

src/moonlink_connectors/src/rest_ingest/moonlink_rest_sink.rs:42

↓ 4 callersMethodappend_row

( &mut self, row: MoonlinkRow, )

src/moonlink/src/storage/mooncake_table/data_batches.rs:146

↓ 4 callersMethodappend_table_row

Append a TableRow from PG copy. Returns an immediately-finished RecordBatch if the buffer is full, otherwise None.

src/moonlink_connectors/src/pg_replicate/initial_copy_writer.rs:107

↓ 4 callersMethodapply_to_batch_with_slice

Similar to [`apply_to_batch`], this function also takes a slice of deletion vector indicated by the [`start_row_idx`].

src/moonlink/src/storage/mooncake_table/delete_vector.rs:89

↓ 4 callersFunctioncheck_directory_file_count

Test util function to get file number under the given directory.

src/moonlink/src/storage/cache/object_storage/test_utils.rs:64

↓ 4 callersFunctioncheck_row_index_nonexistent

Test util function to check the given row doesn't exist in the snapshot indices.

src/moonlink/src/storage/mooncake_table/validation_test_utils.rs:143

↓ 4 callersFunctioncreate_backend_from_base_path

(base_path: String)

src/moonlink_backend/tests/common.rs:197

↓ 4 callersFunctioncreate_file_index

Test util function to create file indices. NOTICE: The util function does write index block file.

src/moonlink/src/storage/table/iceberg/tests.rs:119

↓ 4 callersFunctioncreate_file_index_for_both_batches

( directory: std::path::PathBuf, data_file: MooncakeDataFileRef, start_file_id: u64, )

src/moonlink/src/storage/compaction/test_utils.rs:151

← previous next →301–400 of 2,717, ranked by callers