Chapter 15. Launch Service
The Launch Service provides users with convenient way of remotely executing the CloverETL graphs via a simple web-based interface which can be customized to fit the needs of the users.
The Launch Services can be used with any browser and therefore do not require users to install any software. This allows for convenient control of the graph execution which can be easily tied to external tools if necessary (requests can be sent from custom applications as well).
Launch Service Overview
The architecture of Launch Service is relatively simple and follows the basic design of multi-tiered applications utilizing the browser.
The basic usage scenario of client form page is simple:
User opens the Launch Service web page in his browser
User enters the parameters of the graph (if required)
The data is submitted to the CloverETL Server. This is the moment the service is actually called.
All the results (including error messages or logs) are sent back to the user and displayed in the browser. The logs are also available for inspection via CloverETL Server GUI.
The Launch Service web pages which are presented to users can be fully replaced by client's web application or by third party application which calls CloverETL Launch Service by HTTP request. This allows full customization of the outside appearance of the web - for example, it can be a simple web form which communicates with users in the terminology they are familiar with.
Deploying Graph in Launch Service
To enable users to access the specific graph via Launch Service, several steps have to be taken:
The graph has to be designed to allow its parameters to be passed via dictionary.
The graph has to be configured in CloverETL Server in Launch Service section.
The form which will submit the data to Launch Service has to be written.
Overall the deployment of the graph to the Launch Service is not much more complex compared to the regular graph development process. In following chapters all the steps will be described in more detail alongside some basic examples.
Designing the Graphs for Launch Service
To use the graphs from Launch Service, the Launch Service requires the graph to use dictionary when parameters have to be passed to the graph. Dictionary is a data storage associated with each run of the graph in CloverETL. For more details about the dictionary see section "Dictionary" in CloverETL Designer docs.
To use the Dictionary from the Launch Service, the graph author is required to specify the entries of the dictionary in graph's XML source file. For more details about the Dictionary XML element see section "Dictionary" in CloverETL Designer docs.
Apart from the use of the dictionary, the Launch Service does not impose any other restriction on the graphs it should run. The graphs can therefore use all the facilities provided by the CloverETL engine.
Configuring the Graph in CloverETL Server web GUI
To notify the Launch Service about the graphs that will be available via its interface, the Launch Service has to be properly configured via CloverETL Server GUI.
Launch Service uses launch configurations to store the details about how each graph can be run. Each launch configuration contains full description of the graph's parameters, how they are mapped to the parameters passed from the web interface and so on.
Each launch configuration is identified by its name, user and group restriction. Several configurations with the same name can be created as long as they differ in their user or group restrictions.
Use restrictions can be used to launch different graphs for different users even though they use the same launch configuration (for example, the developers may want to use debug version of the graph while the end customers will want to use the production graph). The user restriction can also be used to prohibit certain users from executing the launch configuration.
Similarly, the group restriction can be used to differentiate graphs based on the group membership of the user which runs the launch configuration.
When the configuration is launched, the correct configuration is picked based on the configuration name, user specification and group specification. If multiple configuration match the current user/group and configuration name, the most specific one is picked (the user name has higher priority than the group name).
Adding New Launch Configuration
New launch configurations can be added by clicking on New launch configuration link on the Launch Services tab in CloverETL Server GUI:
After the configuration has been created it will appear in the table on the left side among the other existing configuration. Before using the configuration user will have to add parameter mapping. To add parameter mappings click on the detail link for the newly created configuration. The details will be displayed on the right side of the window in a simple table:
The Basic Info tab shows the basic details about the launch configuration. These can be modified in the Edit Configuration tab:
Following fields can be modified:
Name - is the name under which the configuration will be accessible from web.
Description - the description of the configuration.
Group - restricts the configuration to specific group of users.
User - restricts the configuration to specified user.
Sandbox - selects the CloverETL Sandbox in which the configuration will be launched.
Graph - selects the graph to run when the configuration is launched.
Save run record - if checked, the details about the launch configuration will be visible in Execution History in the CloverETL Server GUI. If unchecked, the graph executions will not be logged and will not be displayed in the Execution History.
Display error message detail - if checked, detailed error messages will be displayed in case the launch fails. If unchecked, only simpler messages will be displayed to the user.
Finally, the tab Edit Parameters can be used to configure parameter mappings for the launch configuration. The mappings are required for the Launch Service to be able to correctly assign parameters values based on the values sent in the launch request.
To add new parameter mapping click on the New property link. Each property required by the graph has to be created (internal graph properties do not need mappings).
Following fields are available for each property:
Name - the name of the property in the graph's dictionary.
Request parameter - the name of the parameter as specified in the launch request generated by the request page. This name can be different than the name used in graph's dictionary.
Parameter required - if checked the parameter is mandatory and error will be reported if it is omitted.
Pass to graph - if checked the parameter will be also passed to graph among the additional parameters as well as in the dictionary. In such case, the parameter can also be referenced as ${ParameterName} in the graph's XML file. Since the additional parameters are resolved when the XML file is parsed, the graphs which use this method cannot be pooled.
Default value - is the default value which will be applied in case the parameter is omitted in the launch request.
To create the new mapping, click on the Create button after all the fields have been filled. After the mapping is created, it will be displayed in the list of existing mappings. It can be later edited or deleted by clicking on appropriate links.
Sending the Data to Launch Service
To launch the graph which has been configured for use with Launch Service, the user has to send a launch request. The launch request can be sent via HTTP GET or POST methods. A launch request is simply an URL which contains the values of all parameters that should be passed to the graph. The request URL is composed of several parts:
[Clover Context]/launch/[Configuration
name]?[Parameters]
[Clover Context]is the URL to the context in which the CloverETL is running. Usually this is the full URL to CloverETL Server (for example, for CloverETL Demo Server this would be http://server-demo.cloveretl.com:8080/clover).[Configuration name]is the name of the launch configuration which has been specified when the configuration has been created. In our example, this would be set to NewMountains (distinction between upper- and lower-case is important).[Parameters]is the list of parameters the configuration requires in the format used for example by PHP. Therefore the parameter list is a list of name-value pairs separated by "&" character. Each name-value pair is specified as [name]=[value] where value has to be properly encoded according to RFC 1738 to make sure URL is valid.
Based on the above, the full URL of launch request for our example with mountains may be like this: http://server-demo.cloveretl.com:8080/clover/launch/NewMountains?heightMin=4000. In the request above, the value of heightMin property is set to 4000.
Results of the Graph Execution
After the graph's run terminates, the results are sent back from the engine to the server and finally to the user. The output is partially defined in the dictionary which is declared in the graph's XML file. The dictionary can mark selected parameters as output parameters. All the output parameters are sent to the user after the graph execution is finished.
Depending on the number of output parameters, the following output is sent to user:
No output parameters - only summary page is displayed to the user. The format of the summary page cannot be customized. The page will contain details like when the graph was started, when it finished, user name and so on.
One output parameter - in this case the output is sent to the user with its content type defined by the property type in the dictionary.
Multiple output parameters - in this case each output parameter is sent to user a part of multipart response. The content type of the response is either multipart/related or multipart/x-mixed-replace depending on the target browser (the browser detection is of course fully automatic). The multipart/related type is used for browsers based on Microsoft Internet Explorer, the multipart/x-mixed-replace is sent to browsers based on Gecko or Webkit.
Launch requests are recorded in the log files in directory specified by
launch.log.dir property in CloverETL Server configuration.
For each launch configuration one log file named [Configuration
name]#[Launch ID].log is created. For each launch request this file will contain
only one line with following tab-delimited fields:
If the property launch.log.dir is not specified,
log files are created in temp directory [java.io.tmpdir]/cloverlog/launch.
Where "java.io.tmpdir" is system property.
Launch start time
Launch end time
Logged-in user name
Run ID
Execution status FINISHED_OK, ERROR or ABORTED
IP Address of the client
User agent of the HTTP client
Query string passed to the Launch Service (full list of parameters of the current launch)
In case the configuration is not valid, the same launch details are saved into the _no_launch_config.log file in the same directory. All unauthenticated requests are saved to the same file as well.