MCPcopy Create free account
hub / github.com/google-research/language / tf_example_to_structure

Function tf_example_to_structure

language/common/utils/nest_utils.py:71–105  ·  view source on GitHub ↗

Convert a serialized tf.Example into a nested structure of Tensors. Args: serialized_example: String tensor containing a serialized example. structure: A nested structure of placeholders that have specified shapes. Returns: features: A nested structure of Tensors consistent with `s

(serialized_example,
                            structure)

Source from the content-addressed store, hash-verified

69
70
71def tf_example_to_structure(serialized_example,
72 structure):
73 """Convert a serialized tf.Example into a nested structure of Tensors.
74
75 Args:
76 serialized_example: String tensor containing a serialized example.
77 structure: A nested structure of placeholders that have specified shapes.
78
79 Returns:
80 features: A nested structure of Tensors consistent with `structure`.
81 """
82 flat_structure = nest_to_flat_dict(structure)
83 dtype_map = {
84 tf.string: tf.string,
85 tf.int32: tf.int64,
86 tf.int64: tf.int64,
87 tf.float32: tf.float32,
88 tf.float64: tf.float32
89 }
90
91 def _placeholder_to_feature(placeholder):
92 return tf.FixedLenFeature(
93 shape=np.prod(placeholder.shape, dtype=np.int32),
94 dtype=dtype_map[placeholder.dtype])
95
96 flat_feature_spec = {
97 k: _placeholder_to_feature(v) for k, v in flat_structure.items()
98 }
99 flat_features = tf.parse_single_example(serialized_example, flat_feature_spec)
100 flat_features = {
101 k: tf.reshape(tf.cast(flat_features[k], v.dtype), v.shape)
102 for k, v in flat_structure.items()
103 }
104 features = flat_dict_to_nest(flat_features, structure)
105 return features
106
107
108def nest_to_flat_dict(nest):

Callers

nothing calls this directly

Calls 3

nest_to_flat_dictFunction · 0.85
_placeholder_to_featureFunction · 0.85
flat_dict_to_nestFunction · 0.85

Tested by

no test coverage detected

Used in the wild real call sites across dependent graphs

searching dependent graphs…