MCPcopy Create free account
hub / github.com/apache/impala / load

Method load

fe/src/main/java/org/apache/impala/catalog/HBaseTable.java:100–138  ·  view source on GitHub ↗

For hbase tables, we can support tables with columns we don't understand at all (e.g. map) as long as the user does not select those. This is in contrast to hdfs tables since we typically need to understand all columns to make sense of the file at all.

(boolean reuseMetadata, IMetaStoreClient client,
      org.apache.hadoop.hive.metastore.api.Table msTbl, String reason,
      EventSequence catalogTimeline)

Source from the content-addressed store, hash-verified

98 * of the file at all.
99 */
100 @Override
101 public void load(boolean reuseMetadata, IMetaStoreClient client,
102 org.apache.hadoop.hive.metastore.api.Table msTbl, String reason,
103 EventSequence catalogTimeline) throws TableLoadingException {
104 Preconditions.checkNotNull(getMetaStoreTable());
105 Table.LOADING_TABLES.incrementAndGet();
106 try (Timer.Context timer = getMetrics().getTimer(Table.LOAD_DURATION_METRIC).time()) {
107 msTable_ = msTbl;
108 final Timer.Context storageLoadTimer =
109 getMetrics().getTimer(Table.LOAD_DURATION_STORAGE_METADATA).time();
110 List<Column> cols;
111 try {
112 hbaseTableName_ = Util.getHBaseTableName(getMetaStoreTable());
113 // Warm up the connection and verify the table exists.
114 Util.getHBaseTable(hbaseTableName_).close();
115 catalogTimeline.markEvent("Checked HBase table exists");
116 columnFamilies_ = null;
117 // Warm up the connection and verify the table exists.
118 getColumnFamilies();
119 cols = Util.loadColumns(msTable_);
120 } finally {
121 storageMetadataLoadTime_ = storageLoadTimer.stop();
122 }
123 clearColumns();
124 for (Column col : cols) addColumn(col);
125 // Set table stats.
126 setTableStats(msTable_);
127 // since we don't support composite hbase rowkeys yet, all hbase tables have a
128 // single clustering col
129 numClusteringCols_ = 1;
130 loadAllColumnStats(client, catalogTimeline);
131 refreshLastUsedTime();
132 } catch (Exception e) {
133 throw new TableLoadingException("Failed to load metadata for HBase table: " + name_,
134 e);
135 } finally {
136 Table.LOADING_TABLES.decrementAndGet();
137 }
138 }
139
140 @Override
141 protected void loadFromThrift(TTable table) throws TableLoadingException {

Callers

nothing calls this directly

Calls 15

getHBaseTableNameMethod · 0.95
getHBaseTableMethod · 0.95
getColumnFamiliesMethod · 0.95
loadColumnsMethod · 0.95
getTimerMethod · 0.80
setTableStatsMethod · 0.80
loadAllColumnStatsMethod · 0.80
refreshLastUsedTimeMethod · 0.80
getMetaStoreTableMethod · 0.65
closeMethod · 0.65
stopMethod · 0.65
getMetricsMethod · 0.45

Tested by

no test coverage detected