Write dataframe into JSON text files This utilises ``pandas.DataFrame.to_json()``, and most parameters are passed through - see its docstring. Differences: orient is 'records' by default, with lines=True; this produces the kind of JSON output that is most common in big-data app
(
df,
url_path,
orient="records",
lines=None,
storage_options=None,
compute=True,
encoding="utf-8",
errors="strict",
compression=None,
compute_kwargs=None,
name_function=None,
**kwargs,
)
| 18 | |
| 19 | |
| 20 | def to_json( |
| 21 | df, |
| 22 | url_path, |
| 23 | orient="records", |
| 24 | lines=None, |
| 25 | storage_options=None, |
| 26 | compute=True, |
| 27 | encoding="utf-8", |
| 28 | errors="strict", |
| 29 | compression=None, |
| 30 | compute_kwargs=None, |
| 31 | name_function=None, |
| 32 | **kwargs, |
| 33 | ): |
| 34 | """Write dataframe into JSON text files |
| 35 | |
| 36 | This utilises ``pandas.DataFrame.to_json()``, and most parameters are |
| 37 | passed through - see its docstring. |
| 38 | |
| 39 | Differences: orient is 'records' by default, with lines=True; this |
| 40 | produces the kind of JSON output that is most common in big-data |
| 41 | applications, and which can be chunked when reading (see ``read_json()``). |
| 42 | |
| 43 | Parameters |
| 44 | ---------- |
| 45 | df: dask.DataFrame |
| 46 | Data to save |
| 47 | url_path: str, list of str |
| 48 | Location to write to. If a string, and there are more than one |
| 49 | partitions in df, should include a glob character to expand into a |
| 50 | set of file names, or provide a ``name_function=`` parameter. |
| 51 | Supports protocol specifications such as ``"s3://"``. |
| 52 | encoding, errors: |
| 53 | The text encoding to implement, e.g., "utf-8" and how to respond |
| 54 | to errors in the conversion (see ``str.encode()``). |
| 55 | orient, lines, kwargs |
| 56 | passed to pandas; if not specified, lines=True when orient='records', |
| 57 | False otherwise. |
| 58 | storage_options: dict |
| 59 | Passed to backend file-system implementation |
| 60 | compute: bool |
| 61 | If true, immediately executes. If False, returns a set of delayed |
| 62 | objects, which can be computed at a later time. |
| 63 | compute_kwargs : dict, optional |
| 64 | Options to be passed in to the compute method |
| 65 | compression : string or None |
| 66 | String like 'gzip' or 'xz'. |
| 67 | name_function : callable, default None |
| 68 | Function accepting an integer (partition index) and producing a |
| 69 | string to replace the asterisk in the given filename globstring. |
| 70 | Should preserve the lexicographic order of partitions. |
| 71 | """ |
| 72 | if lines is None: |
| 73 | lines = orient == "records" |
| 74 | if orient != "records" and lines: |
| 75 | raise ValueError('Line-delimited JSON is only available with orient="records".') |
| 76 | kwargs["orient"] = orient |
| 77 | kwargs["lines"] = lines and orient == "records" |
no test coverage detected