Function write

python/pathway/io/kinesis/__init__.py:180–403

Streams ``table`` into an `AWS Kinesis stream `_. The connection settings are retrieved from the environment. Args: table: The table to write. stream_name: The Kinesis stream where data will be writte

(
    table: Table,
    stream_name: str | ColumnReference,
    *,
    format: Literal["raw", "plaintext", "json"] = "json",
    partition_key: ColumnReference | None = None,
    data: ColumnReference | None = None,
    name: str | None = None,
    sort_by: Iterable[ColumnReference] | None = None,
)

@check_arg_types
@trace_user_frame
def write(
    table: Table,
    stream_name: str | ColumnReference,
    *,
    format: Literal["raw", "plaintext", "json"] = "json",
    partition_key: ColumnReference | None = None,
    data: ColumnReference | None = None,
    name: str | None = None,
    sort_by: Iterable[ColumnReference] | None = None,
) -> None:
    """
    Streams ``table`` into an
    `AWS Kinesis stream <https://docs.aws.amazon.com/streams/latest/dev/introduction.html>`_.
    The connection settings are retrieved from the environment.

    Args:
        table: The table to write.
        stream_name: The Kinesis stream where data will be written. This can be a
            specific stream name or a reference to a column whose values will be used
            as the stream for each message. If using a column reference, the column must
            contain string values.
        format: Format in which the data is put into Kinesis. Currently ``"json"``,
            ``"plaintext"``, and ``"raw"`` are supported. If the ``"raw"`` format is selected,
            ``table`` must either contain exactly one binary column that will be dumped as it is into the
            Kinesis record, or the reference to the target binary column must be specified explicitly
            in the ``data`` parameter. Similarly, if ``"plaintext"`` is chosen, the table must consist
            of a single column of the string type, or the reference to the target string column
            must be specified explicitly in the ``data`` parameter.
        partition_key: Reference to the column used as the partition key in the produced message.
            It can have any data type, because if it is not a string, the Pathway Live Data Framework will obtain its
            string representation and use that value. Note that the maximum length of a partition key
            in Kinesis is 256 bytes. If the key is not specified, internal row key will be used.
        data: Reference to the column that should be used as data in the produced message in
            ``"plaintext"`` or ``"raw"`` format. It can be deduced automatically if the table
            has exactly one column. Otherwise it must be specified directly. It also has to be
            explicitly specified, if ``partition_key`` is set. The type of the column must
            correspond to the format used: ``str`` for the ``"plaintext"`` format and ``binary``
            for the ``"raw"`` format. Note that the maximum length of one message payload in
            Kinesis is 1 MiB.
        name: A unique name for the connector. If provided, this name will be used in
            logs and monitoring dashboards.
        sort_by: If specified, the output will be sorted in ascending order based on the
            values of the given columns within each minibatch. When multiple columns are provided,
            the corresponding value tuples will be compared lexicographically.

    Returns:
        None

    Example:

    The local test setup is the same as for the case of the input connector: you need
    a way to run Kinesis on your machine. You can use the Docker image
    `instructure/kinesalite <https://hub.docker.com/r/instructure/kinesalite/>`_ to spawn
    it.

    Start the container as follows:

    .. code-block:: bash