StoreFuncs take records from Pig's processing and store them into a data store. Most frequently this is an HDFS file, but it could also be an HBase instance, RDBMS, etc.
| 35 | * this is an HDFS file, but it could also be an HBase instance, RDBMS, etc. |
| 36 | */ |
| 37 | @InterfaceAudience.Public |
| 38 | @InterfaceStability.Stable |
| 39 | public interface StoreFuncInterface { |
| 40 | |
| 41 | /** |
| 42 | * This method is called by the Pig runtime in the front end to convert the |
| 43 | * output location to an absolute path if the location is relative. The |
| 44 | * StoreFuncInterface implementation is free to choose how it converts a relative |
| 45 | * location to an absolute location since this may depend on what the location |
| 46 | * string represent (hdfs path or some other data source). |
| 47 | * The static method {@link LoadFunc#getAbsolutePath} provides a default |
| 48 | * implementation for hdfs and hadoop local file system and it can be used |
| 49 | * to implement this method. |
| 50 | * |
| 51 | * @param location location as provided in the "store" statement of the script |
| 52 | * @param curDir the current working direction based on any "cd" statements |
| 53 | * in the script before the "store" statement. If there are no "cd" statements |
| 54 | * in the script, this would be the home directory - |
| 55 | * <pre>/user/<username> </pre> |
| 56 | * @return the absolute location based on the arguments passed |
| 57 | * @throws IOException if the conversion is not possible |
| 58 | */ |
| 59 | String relToAbsPathForStoreLocation(String location, Path curDir) throws IOException; |
| 60 | |
| 61 | /** |
| 62 | * Return the OutputFormat associated with StoreFuncInterface. This will be called |
| 63 | * on the front end during planning and on the backend during |
| 64 | * execution. |
| 65 | * @return the {@link OutputFormat} associated with StoreFuncInterface |
| 66 | * @throws IOException if an exception occurs while constructing the |
| 67 | * OutputFormat |
| 68 | * |
| 69 | */ |
| 70 | OutputFormat getOutputFormat() throws IOException; |
| 71 | |
| 72 | /** |
| 73 | * Communicate to the storer the location where the data needs to be stored. |
| 74 | * The location string passed to the {@link StoreFuncInterface} here is the |
| 75 | * return value of {@link StoreFuncInterface#relToAbsPathForStoreLocation(String, Path)} |
| 76 | * This method will be called in the frontend and backend multiple times. Implementations |
| 77 | * should bear in mind that this method is called multiple times and should |
| 78 | * ensure there are no inconsistent side effects due to the multiple calls. |
| 79 | * {@link #checkSchema(ResourceSchema)} will be called before any call to |
| 80 | * {@link #setStoreLocation(String, Job)}. |
| 81 | * |
| 82 | |
| 83 | * @param location Location returned by |
| 84 | * {@link StoreFuncInterface#relToAbsPathForStoreLocation(String, Path)} |
| 85 | * @param job The {@link Job} object |
| 86 | * @throws IOException if the location is not valid. |
| 87 | */ |
| 88 | void setStoreLocation(String location, Job job) throws IOException; |
| 89 | |
| 90 | /** |
| 91 | * Set the schema for data to be stored. This will be called on the |
| 92 | * front end during planning if the store is associated with a schema. |
| 93 | * A Store function should implement this function to |
| 94 | * check that a given schema is acceptable to it. For example, it |
no outgoing calls
no test coverage detected