For given NLU ref, returns is_pipe, license_type :return: lang, nlu_ref, nlp_ref, license_type, is_pipe

For every duplicate token in input list, ads N whitespaces for the Nth duplicate

Parse a ISO language identifier from a NLU reference which can be used to load a Spark NLP model_anno_obj

Casting pandas to spark and add index col

Print all NLU components available for a language Spark NLP pointer

Prints all unique component_to_resolve types in NLU

# todo update Print every trainable Algorithm/Model :return: None

Casting str to spark and add index col. This is a bit inefficient. Casting follow # inefficient, str->pd->spark->pd , we can could first pd

Casting str to spark and add index col. This is a bit inefficient. Casting follow # inefficient, str->pd->spark->pd , we can could first pd

Clean storage ref from every str in list

Feature Dependency Resolution Algorithm. For a given pipeline with N components, builds a DAG in reverse and satisfy each of their dependenci

No casting, Spark to Spark. Just add index col

Viz dep result

Infer columns required for ner viz and then viz it. viz_colors : set label colors by specifying hex codes , i.e. viz_colors = {'LOC':'#80008

1) parse pred params from first row maybe 2) serialize/deserialize img

returns a dict where key=col name and value=FinisherExtractorConfig for that col for pipe and finisher. For finisher we need to know

Check if there are any OCR components in the Pipe. If yes, we verify data contains pointer to jsl_folder or image files. If y

Check if there are any OCR components in the Pipe. If yes, we verify data contains pointer to jsl_folder or image files. If y

Return a Word to CHUNK Embedding converter for a given Component. The input cols with match the Sentence Embedder ones The converter is a

1. For Tabla-QA the Question Tapas Col should originate from a doc type -> doc_question -> sent_que

Return a Word to Sentence Embedding converter for a given Component. The input cols with match the Sentence Embedder ones The converter i

add tokenizer to pipe if it is missing :param pipe: pipe :return: Pipe with tokenizer if missing

Returns JSL-Anno-ID and default license type for given anno_class name. Note that an anno which maps to a component with default OS_license, m

Check for embedding_consumer and embedding_producer if they match storage_ref and output level wise wise

Authenticate environment for JSL Licensed models Installs NLP-Healthcare if not in environment detected Either provide path to spark_nlp_for_

Set Secret environ variables for Spark Context

This method takes care that the order of components is the correct in such a way,that the pipeline can be iteratively processed by spark NLP.

This function verifies that every input and output column name of a component_to_resolve is satisfied. If some output names are missi

Check if all dependencies are satisfied.

Check if all dependencies are satisfied.

check if a nlu_ref is pointing to a licensed or open source model_anno_obj. This works by just checking if the NLU ref points to a healthcare mode

Check if OCR/Healthcare lib installed version match up with the secrets provided. If not, this will uninstall the missmaching library :param m

Check if any other component_to_resolve in the pipeline has same storage ref as the input component_to_resolve. Returns 1. If there is a candi

Check for the input component_to_resolve, wether it depends on some embedding. Returns True if yes, otherwise False. :param component

For a Chunk to be added to a pipeline, configure its input/output and set storage ref to amtch the storage ref and enfore storage ref notation

Configure component_list components to output level document. Substitute every occurrence of <document> to <sentence> for every compo

convert all the columns in a pandas df to numpy :param pdf: Pandas Dataframe whose embedding column will be converted to numpy array

Deduct a meaningful name for Embeddings, classifiers, resolvesr, relation extractors, etc.. Will return a dict that maps every Annotator Class

Takes in a list of column names removes the elements which are irrelevant to the current output level. This will be run before return

Generate json with dict contexts at target path

For every NER provider and consumer, enforce that their output col is named <pipe_prediction_output_level>@storage_ref for output_levels word,

For every embedding provider and consumer, enforce that their output col is named <pipe_prediction_output_level>@storage_ref for output_levels