Technical Details
Repos
Padawan consists of several git repos in the group https://code.il2.dso.mil/platform-one/products/bullhorn/padawan
- padawan-sync - the synchronizing code that continually runs to pull site artifacts from the padawan-registry to delete or update site content
- padawan-umbrella - the hosting mechanism (nginx) that serves static site content to users
- padawan-manifests - the kustomize manifests used to deploy the k8s resources and padawan-sync/padawan-umbrella images to staging and production
- padawan-pipeline - the gitlab ci pipelines used by site repos to build/test/deploy their site artifacts to staging and production
- padawan-registry - stores the site artifacts (in GitLab Package Registry) and controls which site versions are deployed to staging and production
Architecture
Architecture Questions
"But what about S3?"
Our initial goal was to use S3 and MinIO to store the static site content and then do a simple sync
to pull the latest site content into the PV. However, technical hurdles exist for decrypting the MinIO credentials inside the Party Bus Pipeline. It was faster to get an MVP up by using GitLab Package Registry. The running pods use a SOPS-encrypted GitLab token for a read-only service account to download the packages.
"Why not a k8s CronJob for the sync?"
We experimented with this initially. However, it would require the PV to be ReadWriteMany, which is not supported by EBS. We could have used EFS, but then Padawan could not be available on the high side.
Certificate to Field
Padawan version 1.x earned a Platform One CtF on 2021-06-24.
Platform One CtF on 2021-06-24
Pipeline Variables
See our Padawan Pipeline Variables page for all the available pipeline variables we can support for your project.
Content Security Policy and Inline Script or Style Tags
Padawan imposes a strictly safe Content Security Policy across all HTTP requests in which default-src, script-src, style-src, image-src, font-src, object-src, media-src, frame-src, and form-action are set to 'self', and frame-ancestors is set to 'none'. To allow flexibility to developers and to allow libraries like Vuetify/React/MaterialUI to build on the platform, each HTTP request will generate a new nonce value that can be used to allow certain inline scripts and inline style tags.
If you do not include this attribute on an inline style/script tag the browser will generate Content-Security-Policy errors to the developer console and the referenced resources will not be loaded. Including this nonce attribute will allow inline tags to work. On each request, Padawan-Umbrella will generate a new random nonce and place this value into the Content Security Policy response header. Padawan-Umbrella will also replace the literal string "PADAWAN_CSP_NONCE" in the outgoing response payload (html/js/etc.) to the random nonce value. To see a demo of the nonce in action visit https://websites.staging.dso.mil/sites/platform-one-products-bullhorn-padawan-examples-csp-nonce-hello-world/ where it shows the nonce value:
This text is successfully being generated by an inline script with the attribute nonce="5GprLVyTSrdZTEzebJfhzGjxHpo17x10"
(source code: https://code.il2.dso.mil/platform-one/products/bullhorn/padawan/examples/csp-nonce-hello-world)
We also have a page to help you troubleshoot your site's CSP issues locally. This helps to construct a set of CSP rules and exceptions that are specific to your site to allow the features you need: Padawan - Debugging a Custom Content Security Policy
INFO
Some static site frameworks, specifically Gatsby, choose to require an insecure CSP. If your framework forces your site to expose a XSS attack vector, we would recommend choosing a different framework. If that recommendation goes unheeded, consult the https://code.il2.dso.mil/platform-one/products/bullhorn/padawan/examples/gatsby-hello-world example project for additional mitigation steps and reach out to the Padawan Help Channel for a CSP exception.
Additional resources on CSP nonce
- https://content-security-policy.com/nonce/
- https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/script-src
Browser-based Routing
By default, Padawan assumes Single Page App (SPA) sites will use hash-based routing (HashRouter in React or the default settings in vue-router for Vue.js). This means that if your site attempts to use browser-based routing your child pages (i.e. /your/child/page
) will result in a 404 when bookmarked, refreshed, or otherwise directly accessed. This is because the site does not actually have a resource at /your/child/page/index.html
since it is custom code in your app that interprets the route /your/child/page
. The fix is to have a custom location defined in the NGINX config (Padawan Umbrella) for your app. Contact the Padawan Help Channel to get the config applied to your site.
With additional configuration, you can utilize browser routing (aka HTML5 History Mode) for your site. See the react-router-hello-world example for more details.
Large Assets
In general it is not a good idea to store large files (i.e. videos) in your site's source code repo. It makes git pulls take longer, increases the pipeline run time, and can hurt the end user performance especially if your site doesn't use caching. As an alternative, you can use https://repo1.dso.mil/platform-one/bullhorn-delivery-static-assets to host your large assets. This is a public repo (no SSO required for read access) that utilizes git-lfs to store large videos, images, pdf files, etc. Contact the Padawan Help Channel if you need to host large assets associated with your site.