Functions — github.com/Mooncake-Labs/moonlink

hub / github.com/Mooncake-Labs/moonlink / functions

Functions2,717 in github.com/Mooncake-Labs/moonlink

⨍ Functions 2,717◇ Types & classes 409

↓ 21 callersFunctionassert_non_evictable_cache_handle_ref_count

Test util function to check non-evictable cache handle reference count.

src/moonlink/src/storage/cache/object_storage/test_utils.rs:110

↓ 21 callersFunctioncreate_table

Util function to create table via REST API.

src/moonlink_service/src/test_utils.rs:609

↓ 21 callersFunctionflush_stream_and_sync_no_apply

( table: &mut MooncakeTable, receiver: &mut Receiver<TableEvent>, xact_id: u32, lsn: Option<u6

src/moonlink/src/storage/mooncake_table/table_operation_test_utils.rs:169

↓ 21 callersFunctionget_disk_files_for_table

Test util function to get disk files for the given mooncake table.

src/moonlink/src/storage/mooncake_table/table_accessor_test_utils.rs:14

↓ 20 callersMethodclient

(&self)

src/moonlink_connectors/src/pg_replicate/tests/test_utils.rs:184

↓ 20 callersFunctioncreate_mooncake_table_and_notify

Test util function to create mooncake table and table notify.

src/moonlink/src/storage/mooncake_table/table_creation_test_utils.rs:525

↓ 20 callersMethodtake_payload

(self)

src/moonlink/src/table_notify.rs:41

↓ 20 callersMethodunreference

(&self)

src/moonlink/src/storage/cache/object_storage/cache_handle.rs:49

↓ 19 callersMethodcommit_transaction_stream_impl

(&mut self, xact_id: u32, lsn: u64)

src/moonlink/src/storage/mooncake_table/transaction_stream.rs:577

↓ 19 callersFunctionsync_mooncake_snapshot

=================================== Operation synchronization function =================================== Test util function to block wait and get i

src/moonlink/src/storage/mooncake_table/table_operation_test_utils.rs:325

↓ 19 callersFunctionverify_files_and_deletions

( data_file_paths: &[String], puffin_file_paths: &[String], position_deletes: Vec<PositionDelete>,

src/moonlink/src/storage/mooncake_table/test_utils.rs:230

↓ 19 callersMethodwrite_object

(&self, object: &str, content: Vec<u8>)

src/moonlink/src/storage/filesystem/accessor/filesystem_accessor.rs:295

↓ 18 callersMethodapply_flush_result

Applies the result of a flush to the snapshot task. Adds the disk slice to `next_snapshot_task`.

src/moonlink/src/storage/mooncake_table.rs:959

↓ 18 callersMethodcreate_iceberg_table_manager

Create iceberg table manager.

src/moonlink/src/table_handler/test_utils.rs:179

↓ 18 callersFunctioncreate_infinite_object_storage_cache

Test util function to create an infinitely large object storage cache.

src/moonlink/src/storage/mooncake_table/cache_test_utils.rs:28

↓ 18 callersFunctioncreate_s3_storage_config

Create a S3 catalog config.

src/moonlink/src/storage/filesystem/s3/s3_test_utils.rs:34

↓ 18 callersFunctiondump_arrow_record_batches

Test util function to dump arrow record batches to local filesystem.

src/moonlink/src/storage/compaction/test_utils.rs:78

↓ 18 callersMethodget_serialized_table_config

(&self)

src/moonlink_backend/tests/common.rs:66

↓ 18 callersFunctionvalidate_new_data_files_and_deletion_vectors_in_snapshot

Validate new snapshot with both new data files and deletion vector created.

src/moonlink/src/storage/table/iceberg/state_tests.rs:387

↓ 17 callersMethodadd_type

(&mut self, name: impl Into<String>)

src/moonlink_connectors/src/pg_replicate/tests/test_utils.rs:200

↓ 17 callersFunctioncreate_test_arrow_schema

()

src/moonlink/src/table_handler/profile_test.rs:400

↓ 17 callersMethoddelete_row

( &self, id: i32, name: &str, age: i32, lsn: u64, xact_id: Opt

src/moonlink/src/table_handler/test_utils.rs:246

↓ 17 callersMethodforce_wal_persistence

Force WAL persistence by sending a periodic WAL event and waiting for completion Note that this assumes that this is being called after an event with

src/moonlink/src/table_handler/test_utils.rs:422

↓ 17 callersMethodload

Relaxed ordering is used here because the counter is only used for internal state tracking, not for synchronization.

src/moonlink/src/storage/mooncake_table/batch_id_counter.rs:33

↓ 17 callersMethodset_readable_lsn

Sets both table commit and replication LSN to the same value. This makes data up to `lsn` potentially readable.

src/moonlink/src/table_handler/test_utils.rs:367

↓ 17 callersMethodset_table_commit_lsn

Directly set the table commit LSN watch channel.

src/moonlink/src/table_handler/test_utils.rs:383

↓ 17 callersFunctionvalidate_not_empty

(field: &str, name: &str)

src/moonlink_backend/src/lib.rs:460

↓ 17 callersFunctionwrite

(writer: &mut W, data: &S)

src/moonlink_rpc/src/lib.rs:41

↓ 16 callersFunctioncreate_test_table_schema

Test util function to get test table schema.

src/moonlink/src/storage/table/iceberg/catalog_test_utils.rs:8

↓ 16 callersFunctionflush_table_and_sync_no_apply

( table: &mut MooncakeTable, receiver: &mut Receiver<TableEvent>, lsn: u64, )

src/moonlink/src/storage/mooncake_table/table_operation_test_utils.rs:137

↓ 16 callersFunctionget_data_files_for_read

(data_file_paths: &[DataFileForRead])

src/moonlink/src/storage/mooncake_table/test_utils.rs:214

↓ 16 callersFunctionget_deletion_puffin_files_for_read

( deletion_vector_puffins: &[NonEvictableHandle], )

src/moonlink/src/storage/mooncake_table/test_utils.rs:221

↓ 16 callersMethodget_wal_config

(&self)

src/moonlink/src/storage/wal/test_utils.rs:105

↓ 16 callersFunctionparse_processed_deletion_log

Test util function to get file id and row idx from process deletion log.

src/moonlink/src/storage/table/iceberg/compaction_tests.rs:102

↓ 16 callersMethodput

(&self, key: K, value: V)

src/moonlink/src/storage/cache/metadata/moka_cache.rs:57

↓ 16 callersMethodset_replication_lsn

Directly set the replication LSN watch channel.

src/moonlink/src/table_handler/test_utils.rs:390

↓ 15 callersFunctioncreate_gcs_storage_config

(warehouse_uri: &str)

src/moonlink/src/storage/filesystem/gcs/gcs_test_utils.rs:21

↓ 15 callersFunctioncreate_mooncake_and_persist_for_data_compaction_for_test

( table: &mut MooncakeTable, receiver: &mut Receiver<TableEvent>, injected_committed_deletion_rows

src/moonlink/src/storage/mooncake_table/table_operation_test_utils.rs:533

↓ 15 callersMethodflush

Flush buffered current content.

src/moonlink/src/storage/async_bitwriter.rs:67

↓ 15 callersMethodget_file_path

(&self)

src/moonlink/src/storage/mooncake_table/snapshot_read_output.rs:32

↓ 15 callersFunctionids_from_state

Extract **all** primary-key IDs referenced in `read_state`.

src/moonlink_backend/tests/common.rs:174

↓ 15 callersFunctionload_arrow_batch

Test util function to load the first arrow batch from the given parquet file. Precondition: caller unit tests persist rows in one arrow record batch a

src/moonlink/src/storage/table/iceberg/test_utils.rs:9

↓ 15 callersMethodmark

Mark the replication position as `lsn` if it is newer than the current value.

src/moonlink/src/lsn_state.rs:35

↓ 15 callersMethodset_directory

(&mut self, directory: PathBuf)

src/moonlink/src/storage/index/persisted_bucket_hash_map.rs:434

↓ 14 callersMethodcreate_namespace

Create a new namespace inside the catalog, return error if namespace already exists, or any parent namespace doesn't exist. TODO(hjiang): Implement p

src/moonlink/src/storage/table/iceberg/file_catalog.rs:338

↓ 14 callersFunctiondefault_rest_catalog_config

()

src/moonlink/src/storage/table/iceberg/rest_catalog_test_utils.rs:29

↓ 14 callersMethoddelete_in_stream_batch

(&mut self, row: MoonlinkRow, xact_id: u32)

src/moonlink/src/storage/mooncake_table/transaction_stream.rs:196

↓ 14 callersMethodget_event_sender

Get the event sender to send events to this handler

src/moonlink/src/table_handler.rs:118

↓ 14 callersMethodget_min_ongoing_flush_lsn

We fallback to u64::MAX if there are no pending flush LSNs so that the LSN is always greater than the flush LSN and the iceberg snapshot can proceed.

src/moonlink/src/storage/mooncake_table.rs:1011

↓ 14 callersFunctionget_unique_file_id_for_flush

(table_auto_incr_id: u64, file_idx: u64)

src/moonlink/src/storage/storage_utils.rs:42

↓ 14 callersMethodread_object

(&self, object: &str)

src/moonlink/src/storage/filesystem/accessor/filesystem_accessor.rs:244

↓ 14 callersMethodsend_event

(&self, event: TableEvent)

src/moonlink/src/table_handler/test_utils.rs:207

↓ 13 callersMethodborrow

(&self)

src/moonlink/src/storage/storage_utils.rs:173

↓ 13 callersMethodclose

(&mut self)

src/moonlink/src/storage/filesystem/accessor/filesystem_accessor_chaos_wrapper.rs:141

↓ 13 callersFunctioncreate_batch_channel

Create a bounded channel for passing Arrow RecordBatches from table copy to writer.

src/moonlink_connectors/src/pg_replicate/initial_copy_writer.rs:43

↓ 13 callersFunctioncreate_test_batch_1

Test util function to dump parquet files to local filesystem.

src/moonlink/src/storage/compaction/test_utils.rs:52

↓ 13 callersFunctiondefault_accessor_config

()

src/moonlink/src/storage/table/iceberg/rest_catalog_test_utils.rs:21

↓ 13 callersMethoddrop_table

Drop a table from the catalog.

src/moonlink/src/storage/table/iceberg/file_catalog.rs:544

↓ 13 callersFunctiondump_deletion_vector_puffin

Test util functions to dump deletion vector puffin file to local filesystem. Precondition: rows to delete are sorted in ascending order.

src/moonlink/src/storage/compaction/test_utils.rs:173

↓ 13 callersMethodfetch_table_schema

( &self, src_table_id: Option<SrcTableId>, table_name: Option<&str>, publicati

src/moonlink_connectors/src/pg_replicate/postgres_source.rs:174

↓ 13 callersMethodfind_record

(&self, raw_record: &RawDeletionRecord)

src/moonlink/src/storage/index/mem_index.rs:5

↓ 13 callersMethodget_operator

Get IO operator from the catalog.

src/moonlink/src/storage/filesystem/accessor/filesystem_accessor.rs:82

↓ 13 callersFunctionget_table_unique_table_id

Test util function to get unique table file id for the given file id.

src/moonlink/src/storage/compaction/tests.rs:41

↓ 13 callersFunctionget_test_row_1

============================== Row preparation functions ============================== Test util functions to get a few moonlink rows for testing.

src/moonlink/src/storage/table/iceberg/state_tests.rs:60

↓ 13 callersFunctionget_test_schema

Test util function to get iceberg schema,

src/moonlink/src/storage/table/iceberg/file_catalog_test_utils.rs:26

↓ 13 callersMethodsanitize_path

Sanitize given path. Opendal works on relative path, so attempt to sanitize absolute path to relative one if applicable.

src/moonlink/src/storage/filesystem/accessor/filesystem_accessor.rs:77

↓ 13 callersMethodsend_table_event

Send a table event to the appropriate table handler (internal helper)

src/moonlink_connectors/src/rest_ingest/moonlink_rest_sink.rs:304

↓ 12 callersMethodappend

======================= Table state updates ======================= The following events contains updates to the mooncake table, which will be record

src/moonlink/src/storage/mooncake_table.rs:1212

↓ 12 callersFunctionassert_is_same_schema

(lhs: IcebergSchema, rhs: IcebergSchema)

src/moonlink/src/storage/table/iceberg/schema_utils.rs:11

↓ 12 callersMethodbuild_from_flush

================================ Build from flush ================================

src/moonlink/src/storage/index/persisted_bucket_hash_map.rs:486

↓ 12 callersFunctioncheck_deleted_rows

Test util function to check the given referenced arrow batch are equal to rows. - [`rows`] are ordered in the order of first element - [`arrow_batches

src/moonlink/src/storage/table/iceberg/compaction_tests.rs:131

↓ 12 callersFunctioncheck_deletion_vector_consistency

Test util function to check consistency for snapshot batch deletion vector and deletion puffin blob.

src/moonlink/src/storage/mooncake_table/validation_test_utils.rs:27

↓ 12 callersMethodcommit_transaction_stream

Commit a transaction stream. - Flushes any remaining rows from stream mem slice - Adds all in mem batches and indices to next snapshot task - Updates

src/moonlink/src/storage/mooncake_table/transaction_stream.rs:683

↓ 12 callersFunctioncreate_test_table_metadata_with_config

Test util function to create mooncake table metadata with mooncake table config.

src/moonlink/src/storage/mooncake_table/table_creation_test_utils.rs:260

↓ 12 callersMethodflush

Flush buffered entries written to disk.

src/moonlink/src/storage/index/persisted_bucket_hash_map.rs:383

↓ 12 callersFunctionget_arrow_batches_with_row_idx

Test util function to get arrow batches in the given rows.

src/moonlink/src/storage/table/iceberg/compaction_tests.rs:114

↓ 12 callersMethodget_lookup_key

(&self, row: &MoonlinkRow)

src/moonlink/src/row/moonlink_row.rs:334

↓ 12 callersMethodlayer

(&self, inner: A)

src/moonlink/src/storage/filesystem/accessor/filesystem_accessor_chaos_wrapper.rs:29

↓ 12 callersFunctionnonunique_ids_from_state

Extract counts for possibly non-unique primary-key IDs referenced in `read_state`.

src/moonlink_backend/tests/common.rs:183

↓ 12 callersMethodperform_wrapper_function

Attempt injected delay and error.

src/moonlink/src/storage/filesystem/accessor/chaos_generator.rs:63

↓ 12 callersMethodprocess_cdc_event

( &mut self, event: CdcEvent, )

src/moonlink_connectors/src/pg_replicate/moonlink_sink.rs:198

↓ 12 callersMethodset_files

(&mut self, files: Vec<MooncakeDataFileRef>)

src/moonlink/src/storage/index/persisted_bucket_hash_map.rs:439

↓ 12 callersMethodset_persistence_snapshot_res

Set iceberg snapshot flush LSN, called after a snapshot operation.

src/moonlink/src/storage/mooncake_table.rs:693

↓ 12 callersFunctiontest_row_1

Test util functions to create moonlink rows.

src/moonlink/src/storage/table/iceberg/tests.rs:134

↓ 12 callersMethodtry_create_mooncake_snapshot

(&mut self, opt: SnapshotOption)

src/moonlink/src/storage/mooncake_table.rs:1506

↓ 12 callersMethodunreference_and_delete

(&self)

src/moonlink/src/storage/cache/object_storage/cache_handle.rs:56

↓ 12 callersFunctionvalidate_state_3

Validate state 3: remote, local, in use

src/moonlink/src/storage/mooncake_table/data_file_state_tests.rs:367

↓ 12 callersFunctionverify_file_contents

( file_path: &String, expected_ids: &[i32], expected_row_count: Option<usize>, )

src/moonlink/src/storage/mooncake_table/test_utils.rs:151

↓ 11 callersFunctionassert_data_and_puffin

Util function to check data file and puffin. Called after insert/upload of a payload/parquet file.

src/moonlink_service/src/test_utils.rs:777

↓ 11 callersFunctioncreate_random_string

Test util function to generate random string with the requested size.

src/moonlink/src/storage/filesystem/accessor/test_utils.rs:9

↓ 11 callersFunctioncreate_table_and_iceberg_manager_with_data_compaction_config

Similar to [`create_table_and_iceberg_manager`], but it takes data compaction config.

src/moonlink/src/storage/mooncake_table/table_creation_test_utils.rs:397

↓ 11 callersMethodextract_identity_columns

(&self, mut row: MoonlinkRow)

src/moonlink/src/row/moonlink_row.rs:304

↓ 11 callersMethodget_mooncake_table_id

(&self)

src/moonlink/src/storage/wal.rs:63

↓ 11 callersFunctionint32_schema

()

src/moonlink_connectors/src/pg_replicate/initial_copy_writer.rs:221

↓ 11 callersFunctionload_one_arrow_batch

Test util function to load one arrow batch from the given local parquet file.

src/moonlink/src/table_handler/test_utils.rs:624

↓ 11 callersFunctionmake_batch

(values: &[i32])

src/moonlink_connectors/src/pg_replicate/initial_copy_writer.rs:225

↓ 11 callersFunctionmeta

Helper to build metadata with a specific field id

src/moonlink_service/src/test_utils.rs:341

↓ 11 callersMethodsearch_values

( &self, value_and_hashes: &[(u64, u64)], )

src/moonlink/src/storage/index/persisted_bucket_hash_map.rs:276

↓ 11 callersMethodsync_snapshot

( &mut self, mut snapshot_payload: PersistenceSnapshotPayload, file_params: Persistenc

src/moonlink/src/storage/table/iceberg/iceberg_table_manager.rs:194

↓ 10 callersMethodbatch_ingest

Batch ingestion the given [`parquet_files`] into mooncake table. TODO(hjiang): 1. Record table events. 2. It involves IO operations, should be placed

src/moonlink/src/storage/mooncake_table/batch_ingestion.rs:71

← previous next →101–200 of 2,717, ranked by callers