note: for legacy projects refer to "taskmanager readme" document.
Introduction
A diverse set of automation types are supported. You can either use single action automation or use third applications that proffer automation functionality. Generally speaking you should use the automation framework when you have a process that:
- generates output that needs to be persisteed
- runs need to be tracable for further review
- you need to specifcy a heterogenous target environment
Example Integrations
Interactive Dashboard for the Izy-Proxy CLI
Generally speacking these items will generate the following tasks
- runtimeid: would be in the "default" or "current browser" environment.
- execution model: service call
- scheduling: manual (initiated by the user)
- stateful: no
- execution tracking: no
The dashboard input will be provided as the $cmd
variable. The Action field will specify either a static module:
module?fn
Or a third party package
node_modules/izyware-dataconsole-elasticsearch/delete?byId
Data Scraping and Change Detection
A typical flow in the entry point module would be:
['net.httprequest', {
method: 'POST',
url: 'end_point',
body: JSON.stringify({ })
}],
['assert.value', {
contextName: 'my context',
headers: { 'content-type': 'application/json; charset=utf-8' },
status: 200,
responseText: JSON.stringify({"success":true})
}],
['task.progress', 10],
You can use the state variable to detect changes. Note that it is recommanded that:
- includes the IDs of the objects, processing state, and result pointers
- hash of the payload should be stored for detecting dups
- dups should be allowed. This will ensure that the reruns and the end to end process is repeatable.
For full white papers on these implementation refer to apps/smtprelay/V documentation.
Device Manager
You can utilize the the ping data from the device manager to automate tasks. Refer to the related documentation for this. The current implementation implements the following states:
- queued: the user marked the task for running, waiting for next ping from the client tab
- waiting for runner: ping api instructed the client tab to run the apploder.js script.
- capturing state: apploder.js is running is about to query the state
Data Model
* name
* description
* scheduling and automatic running (rerun feature)
* manual/automatic, period (how often to run)
* currentClusterGroupId: the current taskrun group id.
It is used to determine the current running state, etc.
* enable/disable: needed for pausing running tasks specially the ones with rerun features
* execution tracking: progress, durtaion, outcome tracking and historical storage
* runningstate: dormant, queued for pickup, running -> dormant
* parametersJSON:
* This will get injected into the entrypointmodule by the framework.
* This value cannot be modified by the task and is set from the admin dashboard or the API.
* Examples of this value:
* Static configuration
* Credentials
* API keys
* stateJSON(optionl): Can be used to store states that might be useful for periodic task.
* Examples of this value:
* last ID retrieved
* time span synced
* etc.
* This value can and should be modified by the task.
* runtimeid: Determines the hosting environment for the task.
* Examples
* url address for the service
* arn for the node inside the izycloud
* runningstate: created|queued|running|complete
* exitcode: success:0,1,2,
* reason string
* starttime
* endtime
* logid: pointer to the logfile on S3
* taskId
* clusterGroupId
* parametersJSON
* stateJSON
Implementing a task runner
The reference implementation for the taskrunner can be found at: izy-proxy/taskrunner/process.js. In case you need to implement your own customer task runner, you can use the task management APIs. The following end-points are available:
Taskrunner API Interface
- apiPakcageName:readList: Gets the list of "queued" tasks for your runtimeid and returns a taskToRun instance object
- apiPakcageName:crud?progress: Set progress and reason for the task.
- apiPakcageName:crud?taskOutcome: Record the execution outcome of the task.
- apiPakcageName:run?queueForReRunIfEnabled
As it relates to using the legacy taskrunner, the current latest apiPakcageName is apps/tasks/5/api
.
Scripting your own task runner
The latest version is
apps/tasks/7:api
. You can use this inside a CLI script:
await modtask.newChainAsync([
['runpkg.setSession', { authorizationToken }],
['//izyware/apps/tasks/7:api?queueForRunning', { taskIds }]
]);
Known Bugs and Issues
- Add execution model type to task object
* async worker: the task gets added and the "worker" will pick it up
* service call: lunched directly and hitting a service end-point
- the error "parseProjectCard cannot get queryObject from action parameters. Please check the parameters. The error was: queryObject is not defined" should be handled gracefully in the terminal window with actionable link to correction.
- add options to cards for call signature prefix
* Named JSON IO (//service/...) signature.
* Direct JSON I/O (adds /apigateway/) -- this is the default
* RAW HTTP
* will have options
- migration for "Filter Proxy" to unified
* replace route with "individualroutingrule"
* After this is done we can remove individualproxy and individualroutingrule item type
- add (transport agent) configuration option.
- The card click should expose additional "placeholder" tabs for backwards compat with task manager
* new tab for statemanagement
* new tab for history
* new tab for events (to cover rerun, etc.)
* device manager / triggers / cptools pipeline should be automated on a rerun loop
* pulls visual data from two chrome device services
* be able to define data "destination" for each device
* share with a single account.
- (legacy taskmanager) Rename parameters to runtime config
* UX should allow the user to pick a module from a module picker
* also check boxes for things like forcepackagereload, etc.
- Tool for tracking live taskrunners
Links
Changelog
V7.1
- 71000001: apps/tasks/7:api clone with security updates
V5.6
- 56000002: update link to the extension app from automation-enterprise to the settings.
- 56000001: apps/tasks/5/api?crud fix bug where sql/jsnode import is missing
V5.5
- 55000003: apps/tasks/5/api?readList allow queryObject.alsoCaptureInProgress
* extends the usecase to multi client and asynchronous implementations
- 55000002: apps/tasks/5/api - implement direct end-point for capturing progress and reason
* similar to chain item: task.progress
* add the ability to set the reason and progress at the same time
- 55000001: implement support for shadowstatus members
V5.4
- 54000011: (apps/tasks/5/api) update service access patterns to V5
- 54000010: card scale on hover with transition
- 54000009: allow clicking on card header per customer focus group feedback
- 54000008: add audio file playback and sample file.
- 54000007: implement systemstatus, startup and shutdown
- 54000006: enforce auth before audiosync setup
- 54000005: implement audio sync ui
- 54000004: implement bulk delete context menu
- 54000003: update search parameters
- 54000002: remove shortdescription field
V5.3
- 53000111: add name and description fieldset
- 53000110: add configuration tab
- 53000109: use updated signature for cli?runWithMethod
- 53000108: Support //execusioncontext/module?fn format for action
- 53000107: Remove extra call signature prefix data for card processor.
- 53000106: Add help comments to new item wizard.
- 53000105: Reschema fields to parameters and actions for better cross platform compatibility.
- 53000104: Add > CLI Command Does not set the correct type
* This has been removed since the consolidation effort
- 53000103: Add > CLI Command, CONFIGURATION field is too small
* This has been removed since the consolidation effort
- 53000102: add "description field" for better UI.
- 53000101: Standardize and consolidate filter proxy and cli command automation types with the unified (entry point, configuration, service url schema)
* Filter Proxy and Cli Command are consolidated into 1
* serviceInterceptorsCSV => action
* serviceTransformersCSV => parameters
* update documentation
- 530001: clone "viewer/view" from "viewer/top" for compatibility with modern shells
V5.2
- update the link to install extension by generating at API level
- add individualroutingrule automation type
- add configtypename to UI
- add delete feature
- add context menu and collaboration feature
V5.1
- update sql access pattern to V5 patterns.
- move taskrunner_chain to izy-proxy
* makes error handling and context management easier
* reduces hard dependency from izy-proxy apps/tasks
V5
- create izyproxyterminal and individualproxy
< V5
- Refactor metascope/cloudwatch
* 'assert/serverResponse', only three operators, limited to the server thing
* it should instead use ['chain.importProcessor', 'izy-proxy/test/assert:chain'],
* would make the task app too heavy
* customers will implement custom data trackers
- deprecate task.getparameters
* this is something that will configure the 'taskrunner' and should not be controlled by the task
* a more proper name is runtime parameters
- make task.getstate/setstate schema based
* contentType is provided by the caller
* this will streamline parsing JSON for the state
- add created by flag to track who created the task (i.e. rerun or from dashboard)
- add the capability to enable/disable rerun from the taskrunner level
* get rid of the ability to do this from the task level
- fix bug: isolated registered chain handlers on each task run iteration
- create sample node module projects for task that increments the state and publish it.
- add logs button to the info tab for the task
- logs view shows logs for all instances
- dropdown should allow filtering for each instance
automation-desktop: https://github.com/izyware/automation-desktop