Construct quantized Pool from non-quantized pool stored in file. This method does not load whole non-quantized source dataset into memory so it can be used for huge datasets that fit in memory only after quantization. Parameters ---------- data_path : string or pathlib.Path
(
data_path,
column_description=None,
pairs=None,
graph=None,
delimiter='\t',
has_header=False,
ignore_csv_quoting=False,
feature_names=None,
thread_count=-1,
ignored_features=None,
per_float_feature_quantization=None,
border_count=None,
max_bin=None,
feature_border_type=None,
nan_mode=None,
input_borders=None,
task_type=None,
used_ram_limit=None,
random_seed=None,
log_cout=sys.stdout,
log_cerr=sys.stderr,
**kwargs
)
| 539 | |
| 540 | |
| 541 | def quantize( |
| 542 | data_path, |
| 543 | column_description=None, |
| 544 | pairs=None, |
| 545 | graph=None, |
| 546 | delimiter='\t', |
| 547 | has_header=False, |
| 548 | ignore_csv_quoting=False, |
| 549 | feature_names=None, |
| 550 | thread_count=-1, |
| 551 | ignored_features=None, |
| 552 | per_float_feature_quantization=None, |
| 553 | border_count=None, |
| 554 | max_bin=None, |
| 555 | feature_border_type=None, |
| 556 | nan_mode=None, |
| 557 | input_borders=None, |
| 558 | task_type=None, |
| 559 | used_ram_limit=None, |
| 560 | random_seed=None, |
| 561 | log_cout=sys.stdout, |
| 562 | log_cerr=sys.stderr, |
| 563 | **kwargs |
| 564 | ): |
| 565 | """ |
| 566 | Construct quantized Pool from non-quantized pool stored in file. |
| 567 | This method does not load whole non-quantized source dataset into memory |
| 568 | so it can be used for huge datasets that fit in memory only after quantization. |
| 569 | |
| 570 | Parameters |
| 571 | ---------- |
| 572 | data_path : string or pathlib.Path |
| 573 | Path (with optional scheme) to non-quantized dataset. |
| 574 | |
| 575 | column_description : string, [default=None] |
| 576 | ColumnsDescription parameter. |
| 577 | There are several columns description types: Label, Categ, Num, Auxiliary, DocId, Weight, Baseline, GroupId, Timestamp. |
| 578 | All columns are Num as default, it's not necessary to specify |
| 579 | this type of columns. Default Label column index is 0 (zero). |
| 580 | If None, Label column is 0 (zero) as default, all data columns are Num as default. |
| 581 | If string or pathlib.Path, giving the path to the file with ColumnsDescription in column_description format. |
| 582 | |
| 583 | pairs : string or pathlib.Path, [default=None] |
| 584 | Path to the file with pairs description. |
| 585 | |
| 586 | graph : string or pathlib.Path, [default=None] |
| 587 | Path to the file with graph description. |
| 588 | |
| 589 | has_header : bool, [default=False] |
| 590 | If True, read column names from first line. |
| 591 | |
| 592 | ignore_csv_quoting : bool optional (default=False) |
| 593 | If True ignore quoting '"'. |
| 594 | |
| 595 | feature_names : string or pathlib.Path, [default=None] |
| 596 | Path with scheme for feature names data to load. |
| 597 | |
| 598 | thread_count : int, [default=-1] |