MCPcopy Create free account

hub / github.com/Shopify/ghostferry / functions

Functions1,224 in github.com/Shopify/ghostferry

⨍ Functions 1,224◇ Types & classes 176
↓ 3 callersFunctionNonExistingPaginationKeyColumnError
NonExistingPaginationKeyColumnError exported to facilitate black box testing
table_schema_cache.go:245
↓ 3 callersFunctionNonExistingPaginationKeyError
NonExistingPaginationKeyError exported to facilitate black box testing
table_schema_cache.go:250
↓ 3 callersMethodPaginationKey
()
dml_events.go:79
↓ 3 callersMethodPanicf
(format string, args ...any)
logger.go:23
↓ 3 callersMethodQueryMySQLVersion
()
sqlwrapper/ghostferry_db.go:61
↓ 3 callersMethodReportProgress
()
ferry.go:1057
↓ 3 callersMethodResumableBinlogPosition
()
dml_events.go:81
↓ 3 callersMethodSeedSourceDB
(numberOfRows int)
testhelpers/unit_test_suite.go:78
↓ 3 callersMethodSerialize
(lastKnownTableSchemaCache TableSchemaCache, binlogVerifyStore *BinlogVerifyStore)
state_tracker.go:308
↓ 3 callersMethodSetup
()
testhelpers/integration_test_case.go:48
↓ 3 callersMethodStart
()
sharding/sharding.go:119
↓ 3 callersFunctionUint64Value
(value interface{})
dml_events.go:461
↓ 3 callersMethodUpdate
(updatedConfig UpdatableConfig)
config.go:937
↓ 3 callersMethodUpdateRow
(id int, data string)
test/go/iterative_verifier_test.go:336
↓ 3 callersMethodValidate
()
copydb/config.go:30
↓ 3 callersMethodWaitUntilRowCopyIsComplete
()
testhelpers/integration_test_case.go:86
↓ 3 callersFunctionappendEscapedBuffer
(buffer, value []byte, isJSON bool)
dml_events.go:542
↓ 3 callersFunctionappendEscapedValue
(buffer []byte, value interface{}, column schema.TableColumn)
dml_events.go:411
↓ 3 callersMethodapplyPrefix
(key string)
metrics.go:124
↓ 3 callersMethodcheckConnection
(dbname string, db *sql.DB)
ferry.go:1111
↓ 3 callersMethodcompletedTables
()
test/go/data_iterator_test.go:169
↓ 3 callersMethodensureLogger
()
binlog_streamer.go:66
↓ 3 callersFunctionformatMismatches
(mismatches map[string]map[string][]InlineVerifierMismatches)
inline_verifier.go:454
↓ 3 callersFunctiongetMultiTableMap
()
test/go/table_schema_cache_test.go:399
↓ 3 callersFunctionhandleErr
(err error, metric interface{})
sharding/metrics.go:75
↓ 3 callersMethodkill
test/helpers/ghostferry_helper.rb:388
↓ 3 callersFunctionminimalConfig
minimalConfig creates the smallest valid Config for testing ValidateConfig().
test/go/logger_test.go:488
↓ 3 callersFunctionnewThrottler
()
test/go/lag_throttler_test.go:32
↓ 3 callersFunctionnewThrottlerWithQuery
(query string)
test/go/lag_throttler_test.go:17
↓ 3 callersFunctionpaginationKeyFromEventData
(table *TableSchema, rowData RowData)
dml_events.go:574
↓ 3 callersMethodpickRandomTable
()
testhelpers/data_writer.go:266
↓ 3 callersMethodreloadTables
()
test/go/iterative_verifier_test.go:374
↓ 3 callersFunctionrequestType
(r *http.Request)
control_server.go:417
↓ 3 callersMethodsendMetric
(metric interface{})
metrics.go:110
↓ 3 callersFunctionsetupSingleTableDatabase
==================== Helper methods below ====================
test/go/trivial_integration_test.go:161
↓ 3 callersFunctiontagsToStrings
(tags []ghostferry.MetricTag)
sharding/metrics.go:63
↓ 3 callersFunctionusage
()
sharding/cmd/main.go:17
↓ 3 callersMethodverifyAllEventsInStore
Returns mismatches in the form of db -> table -> paginationKeys
inline_verifier.go:771
↓ 2 callersMethodApplicableDatabases
(dbs []string)
sharding/filter.go:283
↓ 2 callersMethodAssertTenantCopied
()
sharding/testhelpers/unit_test_suite.go:125
↓ 2 callersFunctionAssertTwoQueriesHaveEqualResult
(t *testing.T, ferry *ghostferry.Ferry, sourceQuery string, targetQuery string, args ...interface{})
testhelpers/helper_methods.go:66
↓ 2 callersMethodAssertVerifierErrored
(msg ...string)
test/go/verifier_test.go:122
↓ 2 callersMethodAssertVerifierMatched
()
test/go/verifier_test.go:102
↓ 2 callersMethodAssertVerifierNotMatched
()
test/go/verifier_test.go:112
↓ 2 callersMethodCalculateBatchSize
(rowSize int)
config.go:496
↓ 2 callersMethodEach
(f func(*RowBatch) error)
cursor.go:87
↓ 2 callersMethodEndCutover
(cutoverStart time.Time)
ferry.go:929
↓ 2 callersMethodErroredOrFailed
()
iterative_verifier.go:113
↓ 2 callersMethodEstimateByteSize
()
row_batch.go:29
↓ 2 callersMethodFlushAndBatchByTable
(batchsize int)
iterative_verifier.go:70
↓ 2 callersMethodGetCompressedHashes
GetCompressedHashes compares the source data with the target data to ensure the integrity of the data being copied. The GetCompressedHashes method ch
compression_verifier.go:62
↓ 2 callersMethodGetHashes
(db *sql.DB, schemaName, tableName, paginationKeyColumn string, columns []schema.TableColumn, paginationKeys [
iterative_verifier.go:292
↓ 2 callersMethodGetHashesFromDifferentCollations
(data, from, to string)
test/go/iterative_verifier_collation_test.go:75
↓ 2 callersFunctionGetMd5HashesSql
(schema, table, paginationKeyColumn string, columns []schema.TableColumn, paginationKeys []interface{})
iterative_verifier.go:668
↓ 2 callersMethodGetUint64
For historical reasons, this function ended up being used at two different places: the DataIterator and the DMLEvent (which indirectly is used by the
dml_events.go:53
↓ 2 callersMethodInitialize
Initialize all the components of Ghostferry and connect to the Database
ferry.go:336
↓ 2 callersMethodInitialize
Initialize allocates the event buffer channel and logger so that BufferBinlogEvents can be called safely before Run() starts in its own goroutine. Fe
binlog_writer.go:33
↓ 2 callersMethodInitializeAndValidateConfig
()
copydb/config.go:82
↓ 2 callersMethodIsAlmostCaughtUp
()
binlog_streamer.go:343
↓ 2 callersMethodMarkTableAsCompleted
(table string)
state_tracker.go:246
↓ 2 callersMethodMarshalJSON
()
pagination_key.go:87
↓ 2 callersMethodMarshalJSON
()
pagination_key.go:173
↓ 2 callersMethodNewBatchWriter
()
ferry.go:187
↓ 2 callersFunctionNewBinlogVerifyStore
()
inline_verifier.go:106
↓ 2 callersMethodNewBinlogWriter
()
ferry.go:162
↓ 2 callersFunctionNewCompressionVerifier
NewCompressionVerifier first checks the map for supported compression algorithms before initializing and returning the initialized instance.
compression_verifier.go:215
↓ 2 callersMethodNewCursor
returns a new Cursor with an embedded copy of itself
cursor.go:49
↓ 2 callersMethodNewDataIterator
()
ferry.go:94
↓ 2 callersFunctionNewDefaultLogger
NewDefaultLogger creates a Logger from the active backend's default configuration. Used as a fallback when no logger is provided.
logger.go:145
↓ 2 callersFunctionNewFerry
(config *Config)
copydb/copydb.go:17
↓ 2 callersFunctionNewFerry
(config *Config)
sharding/sharding.go:17
↓ 2 callersMethodNewInlineVerifier
()
ferry.go:223
↓ 2 callersFunctionNewReverifyStore
()
iterative_verifier.go:36
↓ 2 callersMethodProgress
()
ferry.go:962
↓ 2 callersMethodQueryRowContext
(ctx context.Context, query string, args ...interface{})
sqlwrapper/ghostferry_db.go:52
↓ 2 callersMethodRun
Returns a list of results of the size same as the concurrency number. Returns the first error that occurs during the run. Also as soon as a single wor
utils.go:92
↓ 2 callersMethodRun
Spawns the background tasks that actually perform the run. Wait for the background tasks to finish.
ferry.go:632
↓ 2 callersMethodRunStandaloneDataCopy
(tables []*TableSchema)
ferry.go:833
↓ 2 callersFunctionScanByteRow
(rows *sqlorig.Rows, columnCount int)
cursor.go:295
↓ 2 callersMethodSerializeStateToJSON
()
ferry.go:942
↓ 2 callersMethodSetDB
(*sql.DB)
testhelpers/data_writer.go:105
↓ 2 callersMethodSetDataColumnCollation
(charset string)
test/go/iterative_verifier_collation_test.go:92
↓ 2 callersFunctionSetGlobalMetrics
(prefix string, metricsChan chan interface{})
sharding/metrics.go:39
↓ 2 callersMethodSort
(unorderedTables map[*TableSchema]PaginationKey)
data_iterator_sorter.go:38
↓ 2 callersMethodStart
Attach event listeners for Ghostferry components and connect the binlog streamer to the source shard Note: Binlog streaming doesn't start until Run.
ferry.go:571
↓ 2 callersMethodStart
Method override for Start in order to send status to the integration server.
test/lib/go/integrationferry/ferry.go:83
↓ 2 callersMethodStartCutover
()
ferry.go:909
↓ 2 callersMethodStmtFor
(p SqlPreparer, query string)
utils.go:160
↓ 2 callersMethodTableSchema
()
row_batch.go:54
↓ 2 callersMethodUpdateLastResumableBinlogPositionForTargetVerifier
(pos mysql.Position)
state_tracker.go:189
↓ 2 callersMethodUpdateLastResumableSourceBinlogPosition
(pos mysql.Position)
state_tracker.go:175
↓ 2 callersMethodUpdateLastResumableSourceBinlogPositionForInlineVerifier
(pos mysql.Position)
state_tracker.go:182
↓ 2 callersMethodUpdateRowInDb
(id int, data string, db *sql.DB)
test/go/iterative_verifier_test.go:340
↓ 2 callersMethodValidate
()
sharding/config.go:97
↓ 2 callersMethodValuesContainPaginationKey
()
row_batch.go:46
↓ 2 callersMethodWait
()
testhelpers/data_writer.go:138
↓ 2 callersMethodWaitUntilBinlogStreamerCatchesUp
()
ferry.go:863
↓ 2 callersMethodWaitUntilRowCopyIsComplete
Call this method and perform the cutover after this method returns.
ferry.go:859
↓ 2 callersFunctionWithRetriesContext
(ctx context.Context, maxRetries int, sleep time.Duration, logger Logger, verb string, f func() error)
utils.go:24
↓ 2 callersFunctionaddJoinID
(db *sql.DB, dbName, tableName string)
sharding/testhelpers/unit_test_suite.go:186
← previousnext →201–300 of 1,224, ranked by callers