Parse External Data in a Dataset
- A base dataset using external data (from uploading a file or connecting to a server) exists in the Data Catalog.
- Security:
- Prism Datasets: Createdomain in the Prism Analytics functional area when creating a dataset.
- Any of these security requirements when editing an existing dataset:
- Prism Datasets: Managedomain in the Prism Analytics functional area.
- Dataset Editorpermission on the dataset.
- Dataset Ownerpermission on the dataset.
When you bring external data into a base dataset, you must describe the source data in a tabular format. You do this by describing how to parse the data.
If you're new to Workday, you don't have access to create or edit base datasets.
Your data must:
- Be in plain text file format.
- Have 1 record per line.
- Have the same sequence of fields for every record separated by a common delimiter (such as a comma or tab).
Delimited records are separated by line breaks, and fields within a line are separated by a special character called the delimiter (usually a comma or tab character). If the delimiter also exists in the field values, it must be escaped. Base datasets support single character escapes (such as a backslash), as well as enclosing field values in double quotes.
- Access theEdit Dataset Transformationstask for a base dataset using external data.
- Edit the Parse stage.As you complete this task, consider:
Option Description Row DelimiterSpecifies the single character that separates rows (or records) in your source data files.In most delimited files, rows are separated by a new line, such as the line feed character, carriage return character, or carriage return plus line feed. Line feed is the standard new line representation on UNIX-like operating systems. Other operating systems (such as Windows) might use carriage return individually, or carriage return plus line feed. SelectingAny New Linecauses Workday to recognize any of these representations of a new line as the row delimiter.Field DelimiterSpecifies the single character that separates the fields (or columns) of a row in your source data files. Comma and tab are the most common field delimiters.Field NamesSpecifies the default name of each field. You can change the field names in the Parse stage after you finish defining the parsing options.Field names must follow naming validation rules.Workday automatically treats the first line in each source file as a header row instead of as a row of data. If you don't want to use the first line as names for your fields, deselect theField Namescheck box.Escape CharacterSpecifies the single character used to escape theQuote Characteror another instance of theEscape Characterwhen aQuote Characteris specified. Workday reads an escape character as data only if it's escaped with another escape character.If your data values contain quote characters as data, those characters must be escaped and the entire field value must be enclosed with theQuote Character. If not, then Workday assumes that the quote character denotes a new field.For comma-separated values (CSV) files, it's common practice to escape field delimiters by enclosing the entire field value within double quotes. If your source data uses this convention, then you should specify aQuote Character.Quote CharacterSome delimited files use the quote character to enclose individual data values. The quote character is typically the double quote character (").If a field value contains a field delimiter as data, then the field value must be enclosed in theQuote Character, otherwise Workday assumes that the field delimiter denotes a new field.If a field value contains the quote character as data, then the field value must be enclosed in theQuote Characterand it must be escaped, either by theEscape Characteror another quote character.If a field value contains a row delimiter (such as a new line character) as data, then the field value must be enclosed in theQuote CharacterandField values contain new linesmust be selected.Suppose that you have a row with these 3 data values:weekly special wine, beer, and soda "2 for 1" or 9.99 eachIf the field delimiter is a comma, the quote character is a double quote, and the escape character is a backslash, then a correctly formatted row in the source data looks like:"weekly special","wine, beer, and soda","\"2 for 1\" or 9.99 each"Rows to ignoreSpecifies the number of lines at the beginning of the file to ignore when reading the source file while creating and publishing the dataset. To use this with theFrom First Table Rowoption, ensure that the line containing the field names is visible and is the first remaining line.Field values contain new linesCheck this option if your source data might contain new line characters as part of a field value.When enabled, Workday reads the new line characters inside quote characters as part of the field value instead of as a row delimiter. Workday interprets any row delimiter character outside of quote characters as a new record.Enabling this option might impact the time to publish a dataset if Workday reads very large source files.Note that you might get unexpected results if you enable this option and the source file has malformed data (such as when a field value has either an opening or closing quote character, but not both). Try to ensure that your source data is well formed when using this option.Trim trailing and leading whitespace characters in Text fieldsSelect this check box if you want to remove whitespace characters at the beginning and end of Text fields.