izy-circus
Circus is useful for lightweight headless rendering and building sophisticated debuggers for application and A/B testing.
INSTALLATION
Vesion 2.0 and above are deployed as an IzyCloud package which can be referenced as a content engine, for example:
bootstrap/5/circus:http
earlier versions were deployed as nodejs packages. See notes below.
USING THE TOOL
From the IzyCloud dashboard, add the following as the content engine
bootstrap/5/circus:http
This falls into the first bullet item of the main use cases listed below:
- render for search engines and allow running the bootstrap from browser. This will in turn configure izy-proxy.http plugin to process your doman via bootstrap/5/circus:http
* input: intitial url, all the meta urls generated also
* output: content HTML based on URL, image payloads, etc. - 404s, etc.
* notes: config is passed with "component" parameter which assumes "?queryGroups" and status being present in the response. This approach does not use the HTML to extract relationships
* in order to render Angular and React applications you must decorate the application canvas using the izy tags. Refer to the tagging documentation for more information.
- render in the browser and allow interactive IDE
* build a payload that calls bootstrap/7/entrypoint.start(), uses the uiCoreRootPath set to ui/5. - no ComposeConfig specified but config is burned in the image
* input: the current browser url, cookie for session
* output: rendered html
* notes: ui/5 works with IDE and does the queryGroups and supports groups rendering, etc.
- render on the terminal in the dashboard scenario
* xcast/dashboard/viewer/terminal calls bootstrap/7/entrypoint.start() uses the uiCoreRootPath: 'rel:ui/terminal' and dashboardComposeConfig set to './test/workstation
audioout
toadmin_speake.json'
* you may use the shortcut
modtask.ldmod('izy-circus/index').bootstrap({
bootstrapEntrypointModule: 'bootstrap/7/entrypoint',
appModule: 'your_package/dashboard/viewer/view'
});
- cli launch directly via izy-proxy
* simple call. No application lifecycle.
Visualization Utilities
You can tag your Angular or React application's element with
data-izy-circus-inner-html-status
and monitor the values by color coding them using the izyCircusInnerHtmlStatusToANSIColor. Please refer to the demo app in izy-proxy for examples.
Granular User Management Features
The behavior may be customized by using the following usermetadata
keys:
- component (string)
- platformversion (number)
- bootstrapUrl(string)
- generateHtmlContentSection (boolean): When platformversion > 3, the system will by default generate the server side HTML content. Setting this flag to false will disable this behavior. This is useful for experimental accounts where the applications are still slow and it is sometimes troublesome.
Testing
Unit Tests
npm run unittest queryObject.verbose.log true queryObject.testcase ...
node cli.js method chain chain.action "//inline/bootstrap/5/uriframe:test/index" chain.queryObject.verbose true
Integration Tests
Use
npm run integrationtest queryObject.verbose.log true queryObject.testcase ...
These tests feed path strings to the izyware.com shell and test various behaviors. Note that the following:
- the current shell: If the shell path structure changes these tests will need to be updated
- anonymous and
michael.greer@izyware.com
users: these two user profiles are used for testing public and logged in behaviors where generateHtmlContentSection is expected to be on. The Michael Greer
is required for testing non-public core apps (i.e. settings area).
Local Debugging
Refer to the izy-proxy server configs.
Deployment Dependencies
//cloud/bootstrap/@version/circus:
//cloud/bootstrap/@version/sessman:api
//cloud/bootstrap/@version/uriframe:
HTML Warning Comments
In addition to providing performance and diagnostics via HTTP headers, WARNINGS are also logged as html comments. For example:
`
">setMetaData: !pulse.address || !pulse.id is missing. link will be affected !------------>
`
Izy Cloud Watch will scan your pages for warnings like this and alert if there are any content generation issues.
NOTE
for more details, visit https://izyware.com
Current Pacakges:
- bootstrap/@version/circus
Known issues
- bootstrap/7/ui/terminal/parts/html/util.js
* update the kernel/selectors logic to support fileSystemMode.
izy-uri-attachment
tags are not being rendered to images. need to fix that.
- invalidating the session (i.e. by deleting the session file) and refershing will cause 500 error
* log in, delete the session file (see accounts readme) and refresh https://izyware.com/product.
* that is because //izyware/bootstrap/5/sessman:api?whoamifull will generate an error. it should instead handle this gracefully by returning an "invalid" code to the caller and have the caller do 302 or something.
- when running on latest izy-proxy (wonder if these are timing bugs)
* the incognito mode anonymous refresh issue
* weird session bleeding issues (user 2 was used to cache marketing)
* may be just dont allow caching for logged in?
* does this make using e-tags moot
* should accept sequenceNumber, timestamp as optional params.
modtask.objects[i].imgUri += '-5-' + modtask.objects[i].sequenceNumber + '-' + modtask.objects[i].timestamp;
- use package.json and izy-proxy for testing
*
INPROGRESS_
- circus.cachebust is needed when TTL is not appropriate
- add redirect to https into circus and make it configurable
- the browsers do not send the URI fragment back to the server. therefore cut/paste in the address bar will not displaythe correct "metadata"
* this is only an issue for tools like Slack, Facebook, etc.
* possible workarounds are having the option to change url when navigating
* this should be done after performance optimization for the initial load
- fix bootstrap/5/config to grab circus from server environment
- Currently nav/multi implements frame functionality + state and Uri functionality
* snould be broken into the two pieces
* it is already being passed to circus in the bootstrapper as
modframe
- fullState is not supported for root schema
* support this and enforce length of 1
- auth_izywarefrontend need to have a package inclusion list such that only select packages are allowed to be imported
* feature needs to be added to the package manager dashboard
- performance improvement: we already have a cacheManager (see izyware/viewer/state.js)
* so may be the server will populate the packages on the DOM and that cache manager will use that to speed up the app via the releaseEnabled, etc.
- apps/websites/engine/circus:http improvements
* Move the chrome extension to the header so that it loads before the images, etc.
* embedd the initial package load stuff on the page when it comes through
* appload: right now we do 3 round trips,
* load https://izyware.com/chrome_extension.js without caching
* https://izyware.com/chrome_extension.js still uses an internal importpkg mechanism with izyware/viewer/state which uses the index.php still. At least for the
* apps/websites/engine/pulse:http better that izy-circus?
* We should not be using https://izyware.com/chrome_extension.js and instead embed the payload in the HTM
* alternatvie: put framework releasese on via CDN for fast access
Change Log
7.3
- 73000007: make sure raw mode is only attempted when the input is provided by a TTY
* customers have reported failures in daemon environment
./ldo.sh mship bootstrap/7/build;
$SERVER/putfile.sh ~/plat/p/
build/payloads/build/bootstrap/7/entrypoint/apploader.js $HTDOCS/apps/izyware/srv/73/bootstrap/apploader.js
/* todo
./ldo.sh izy.push bootstrap/7/sessman true
update usermetadata set value = 7 where name = "platformversion" and userid = 126 limit 1;
*/
update usermetadata set value = "https://izyware.com/srv/73/bootstrap/apploader.js" where value = "https://izyware.com/srv/7/bootstrap/apploader.js"
- 73000006: implement exlucdeFromFiltering for error.outcome regions
- 73000005: inject izyCircusInnerHtmlStatusToANSIColor via config
- 73000004: implement terminal monitoring for modulePathShort.groupName.groupPropertyName
- 73000003: allow showing groups with larger than 1 member
- 73000002: enhance terminal visualization by leveraging aria and data attributes on elements
* allow associating data- and aria- attributes to terminal feedback elements like color and audio.
* use izyCircusInnerHtmlStatusToANSIColor to map data-izy-circus-inner-html-status to terminal colors
- 73000001: implement instrumentation analytics data customization
* bootstrap render will process filter, fieldsSchema, fields consistent with izy-proxy DSL (Domain Specific Language ) consistent with elasticsearch
* support for exists operator in the fields
7.2
- 72000009: import izy-circus/bootstrap/7/theme/companyportal
- 72000008: allow simple bootstrap method link to the latest platform version
- 72000007: allow overriding the group module by useCustomModuleForGroupOperations
- 72000006: import bootstrap/7/ui/terminal
- 72000005: implement dom emulation for ui/terminal
* implement terminal/basic and terminal/base
* useCustomModuleForGroupOperations with ui/terminal/host/groups
* temporarily implement fileSystemMode for getResource
- 72000004: allow customization of host/browser/base through config for terminal support. improve build process
* remove izy-proxy prefix in all module paths and add izy-proxy to default path
* remove prefix for audio libs in paths
* replace all direct references to document and window objects to the base module
* implement useCustomModuleForGroupOperations to overwrite default group operations in terminal
- 72000003: initial migration ui/5 to bootstrap/7/ui/terminal
- 72000002: bootstrap/7 add service and editor font size changes, streaming and networking components
7.0
- 70000005: bootstrap/7/ide - increase font size to 16
* hacky solution since changes in ui/ide/editor/ace
package would not show up
- 70000004: bootstrap/7/ide - update links from ide/5 to bootstrap/7/ide
- 70000003: bootstrap/7/ide - update links from ide/5 to bootstrap/7/ide
- 70000002: clone ide/5 to bootstrap/7/ide
./ldo.sh izy.push bootstrap/7/ide true
- 70000001: fork boostrap 7 from 5
./ldo.sh mship bootstrap/7/build;
$SERVER/putfile.sh ~/plat/p/
build/payloads/build/bootstrap/7/entrypoint/apploader.js $HTDOCS/apps/izyware/srv/7/bootstrap/apploader.js
./ldo.sh izy.push bootstrap/7/sessman true
update usermetadata set value = 7 where name = "platformversion" and userid = 126 limit 1;
insert into usermetadata (tscreated,tschanged,userId,name,value) values (utctimestamp, utctimestamp, 126, "bootstrapUrl", "https://DOMAIN/srv/7/bootstrap/apploader.js");
5.4
- 540000006: reorganize tests and standardize folder and file structure
- 540000005: handle /metagateway/address-shardIdID-itemType-sequenceNumber-timestamp url format
- 540000004: handle /metagateway/address-shardIdID-itemType-sequenceNumber url format
- 540000003: should handle /metagateway/test-content-1F4421AB816-3 gracefully
- 540000002: load images from izyware.storagedata on itemstoragedata.storagedataid
* needed for multi payload support
- 540001: circus - add izy-proxy package dependency. refactor tests.
5.3
- 53000479: add support for object schema.type
* allows passing generic objects around easier and via the href links
- 53000478: izy-prozy/izymodtask regression: replace encodedecode/uri uri
- 530033: serialize linkState values
* allows passing kvsValidated and objects to the values
- 530032: fix linkToHref for .. and add test case linkToHrefUriDotDot
- 530031: add
cacheselect
url and track userAgent when caching
- 530030: add TTL to cache items.
- 530029: forceFrontEndUrl only when generateHtmlContentSection is set.
* so that when logged in, refreshing should preseve the full front-end url.
- 530028: seperate the cachable content from bootstrap and standard headers, etc.
* this will make sure that the bootstrap information does not get cached.
- 530027: add whoamiExecutionContext for help with integration tests
* Now the integration tester can set session and force the whoami to run "inline"
- 530026: sessman: use current session when authorizationToken is not provided.
* useful for testing and sessman is forced to run inline.
- 200325: circus/chain: add support for link.suffix
* This is required for redirecting "public" paths have already been fixed by SEM, etc. to internal path structures, for example when user lands on /help/browse/15
and the initial capture is:
{ workspace: "help/browse/15", app: ""}
This will be redirected to:
{ state: { workspace: "public", app: "help" }, suffix: 'browse/15' }
- 200324: circus/chain: fullState is not supported for root schema
- 200323: circus/chain improve error handling when invalid links are present.
5.2
- do not cache when generateHtmlContentSection is set to false
- sessman: add boolean and null types to usermetadata
* allows definition of true/false boolean values and null values for JSON objects
5.0
* bootstrap/5/uriframe
* matchPath should extract the proper leftoverPath.
- improve hrefUri and make it work on FE
pass in modnav as modframe to the circus. chain handler
* use the current frame to deduce fullpath functionality
- use 'circus.enableUrlMode' at startup
* this will make sure verbose settings are applied
- add versioning to the apploader
* Concurrent releasing of the front-end framework packages is prone to breaking existing functionality
* https://DOMAIN/srv/
version/bootstrap/apploader.js
* It would be better to keep apploader.js seperate and NOT autogenerate via izy-proxy
* this will ensure some stability even when izy-proxy is updated
* on the flip side, there could be disparity between server side and client side if this is not tied to izy-proxy but that can always be solved quickly with a new release
- make the apploader fully configurable
- modernize apploader.js to have parity with izy-proxy
./ldo.sh mship bootstrap/5/build
~/SERVER/putfile.sh
file SERVER/htdocs/apps/izyware/srv/
version/bootstrap/apploader.js
- 200321: double slash in redirect (fix)
- add session config option from usermetadata
* enables shell customization per user
- fix forcepackagereloadhack bug.
- add support for link with fullState = /
* this is needed for cross application linking (i.e. editing markdowns belonging to a collection)
- add /cachedelete url
- getMetaGatewayAddress remove https from links
* allows easy running on http
* use the same protocol as the main page
- serialize: fix bug where itemType was not used for circus.getMetaGatewayAddress
- add caching
* 141 pages would have taken 10 minutes to crawl and would spike the CPU
* page load without caching was about 10 seconds
* outages
* after caching its immediate and page load < 1s
* 141 pages took 2 minutes
- add support for groupsmembers schema
- ignore schemaUriToKVS when it does not exists
* utilizes izy-proxy feature methodnotfoundstatus=methodnotfound
- expanded the nextComponents to allow passing of parameters into the queryContent
- add generateHtmlContentSection config flag
- add the ability to specify platform versions per userId
* this will allow having a running system with existing customers on the old stuff while trying out new things and implementing new apps under new accounts for new customers
* for the first step userplatformversion option was added to the domain config
"userplatformversion":{ "124": 5, "anonymous": 5 }
- get rid of bootstrapFile and use bootstrapUrl for specifying the file
* the file://domain/path schema can be used
- implement extraction of authorizationToken from cookie and settings the session
* this is required for correct functioning of the apps on the backend
* this is also needed for platform versioning per user
- add forcepackagereloadhack
* this will work around the "cannot determine module to package mapping" when running stuff locally
- add disableAutoStart to url config
- use schemaUriToKVS, handle 301s
- enhance framesetschema error handling. deprecate framesetnav
- add new tests for href generation
- improve tests and refactor config out of tests
- have backend also use 'circus.addUriProperties' for consistenty with front-end link
- simplify viewLinks, collectionItems schemas for easier serialization into HTML
- get rid of collectionLinks. use collectionItems link property instead
* this will make generating FE html canvas easier
- implement true full path serialization for addUriProperties.hrefUri
* so both circus and front end hrefs use full path
* will allow full right click on the front end canvas as well.
- created circus.enableUrlMode chain action
- moved the engine to versioned bootstrap directory
- add hrefUri to addUriProperties for anchor elements
- add addUriProperties chain item that can be used by everybody (backend, front-end, etc.) to figure out image uris, ec/
* need to be able to augment collection items with a property for the metaurl
['circus.addUriProperties', { properties: ['imgUri'], items : collectionItems }]
v3
* get rid of custom frame_importpkgs and use izy-proxy implementation inside
- Added auth_izywarefrontend to the auth modes for izycloud pkg import API
* Do not expose the main auth token
- Got rid of using izynode and use izy-proxy components for the bootstrapper
* use samples/pkgloader/izycloud.js for the new bootstrapper and inside the izyware app
* updated izyware/viewer/state.js to use that as an option
* added useIzyCloudPkgLoader flag
* updated build process for chrome_extension.js to not write to the server
chromeextension.js -> httpservice -> nginx
./ldo.sh mship izyware/build
$SERVER/putfile.sh PLATP/build/payloads/build/izyware/appstore/config/apploader.js SERVER_/htdocs/apps/izyware/srv/3/bootstrap/apploader.js
use the izy-proxy/ components for import and runpkg inside the bootstrap (used to be chrome_extension.js) and izyware app
v2
- retired test/lib/assert, izy-circus/test/chainitem/assert and use the izy-proxy assertion librariy instead
- support linking and uri generation for non pulse items
* fixes "Make sure that APPS have the correct image metadata" https://izyware.com/
/dashboard/view/542147/info/314
- make tests consistent with izy-proxy guidelines
- deserialize URL to support utf-8 international characters
- can communicate non 200 statuses from the api callbacks
- since apps/pulse defines the validate api, the following can now be cleanly handled
* redirect for illegal paths
* no longer need white listed base paths to know whether it should 404.
- switch to using kvsSchema instead of direct pathPattern structures
- use api schemas instead of calcValidateState and calcPulses , calcNav, etc. to query the components
* easier to implement signatures using the chains
* universally accessible
* each piece can be tested via the izy-proxy interface
* see also https://izyware.com/
/dashboard/list/542147/info/218 (consolidate into izy-circus readme)
- turn circus into a izy cloud module
* it now can be queired from the chains and referenced as a content module inside the website apps
v1
- added tests for md, sitemap, etc.
* could be done by
`
// going to the izy-circus package
node cli.js method test
`
- refactored the loadPackageIfNotPresent inside izy-circus/render/v3.js to use the configurable pkgloader using izy-proxy 2.0;
- implemented as a node package which could be used by
`
npm install izy-circus
var _circus = require('izy-circus');
var outcome = _circus.factory(config);
`
from izy-proxy local, you can simply configure the package to auto reload and do
`
node cli.js
rsync -av --progress ./* ../izy-proxy/nodemodules/izy-circus --exclude nodemodules
`