Devops Tech

Just Thinkin About DevOps Stuff (and Dune)

“In this time, the most precious substance in the universe is the spice (DevOps)… The spice is vital to (Agility). The (IT Dept) and its (engineers), who the spice has mutated over (10) years, use the spice (Code), which gives them the ability to fold (time).”

DevOps enables high speed evolution.

DevOps eliminates wasted resources.

“Software is Eating the World”, “Everything Is Code”, therefore DevOps is key to all success.

The DevOps Cycle

Design/Plan

• Planning docs should be in version control

TDD (Test Driven Design)

TDD is key to AI Assisted/No Code future. Took me a bit to come around, but GitHub Copilot AI was the last straw because it will clearly someday be able to write code to pass your tests, so ultimately, software development will consist of designing tests using some sort of high level language plus drag-and-drop.

Programmers will write DDD libraries to translate domain definitions and diagrams to generate the test for TDD. The human in the loop can approve or overwrite the autogenerated test conditions.

• Design docs need to clearly define exactly what success looks like in the form of tests and expected results. Written by product managers.
• Eventually, changing DDD and/or TDD code will automatically kick off the entire dev lifecycle. AI will handle implementation.

Architecture Diagrams

Overview: Diagraming Tools For Cloud Infrastructure (redhat)

• Diagrams.net formerly Draw.io
• RedHat Portfolio Arch Center
• Gliffy
• MS Visio
• FigJam Figma team whiteboard
• Mermaid Works on GitHub
• Lucidchart

Diagram As Code

• Structurizr
• PlantUML
• Terrastruct
• Graphviz

UI Design Tools

• Figma
• Adobe XD
• Storybook

Manage Tasks, Issues

• Jira
• Asana
• Azure Boards Part of Azure DevOps Server (was MS TFS).
• Trello Kanban, bought by Atlassian (Jira).
• GitHub Issues
• UrbanCode Velocity
• Confluence
• zendesk
• servicenow

Version Control

Source Code, Designs and Docs

• Git
  • GitHub Owned by Microsoft.
  • GitLab
  • AWS CodeCommit
  • Azure Repos
• Workflows For Devs
  • Branching Strategies
    • Trunk Based. Good for microservices, small teams, closed source.
    • Git Flow Good for monoliths, open source, security centric.
• Workflows for Ops
  • GitOps
  • Git Flow
  • GitHub flow Branch/feature based.
  • Trunk Based
  • Feature Based AKA GitFlow
  • Trunk vs Feature

Artifacts

• Github Package Registry (GPR)
  • NPM, Maven Java. Container Registry moved to GHCR (below)
• Artifactory: JFrog. Universal Artifact Mgt.
• UrbanCode Release
• Azure Artifacts
• Nexus: Sonatype. Repository. Containers.
• Language Specific Artifacts
  • NPM Packages: only for node packages
  • PYPI (Python Package Index)
  • Maven Repo Java. Gradle uses Maven repos. Defaults to Maven Central. Google also has a Maven Repo.
  • GoLang
• Images/Containers
  • Docker Hub: limited to docker images
  • GitHub Container Registry (Docker and OCI) ghcr.io
  • JFrog Container Registry docker, OCI
  • Helm Registries: Roll your own Docker or OCI.

CLI First and Automation

• Everything should be code; therefore software should have codeable API’s or SDKSs so that it can be automatable.
• If an app offers a GUI, try not to use it. Use the CLI.
• If a solution looks good but it does not offer a CLI, an SDK, or at least a REST API, skip it.
• VSCode is my default because it was conceptually built from the ground up to run in a web browser. They made a desktop version while they worked out the pure web kinks. Terminals are first class citizens.
• Learn VI or VIM because they are available on most Linux distros. Nano is easier but less likely to be installed.

Build, Test Coverage

• Hamcrest: Matchers that can be combined to create flexible expressions of intent. Was just Java, now available for most languages.

• Java:

  • Build
    • Apache Maven
    • Gradle Highly scriptable. Uses Groovy language that can embed Java.
  • Test
    • JUnit
    • Mockito
    • TestNG
    • Cucumber
  • Quality
    • JaCoCo: IDE and Jenkins plugin versions available. Emma replacement.
    • JTest

• Python

  • Uses setup.py file (setup script).
  • PyBuilder
  • Security
    • ossaudit
    • Jake BOM

• JS WebApps:

  • Build
    • WebPack
    • Parcel Also supports Webassembly
    • Snowpack: trying to replace WebPack and Parcel. O(1) build times.
    • Rollup.js
    • esbuild The new hotness. Written in GoLang.
    • Packem Written in Rust.
    • Babel: ES transpiler
  • Unit Testing
    • Jest
    • Mocha.js: Test framework/suite. Modular.
    • Chai: TDD and BDD assertion framework. Mocah pluginable.
    • Sinon.js: stub, spy, mock. Standalone or plugin.
    • Jasmine No external dependencies!
  • Security
    • AuditJS
    • Sonatype Nexus IQ Extension for VSCode

• GoLang: builtin. Lean Go with Tests

  • Security
    • Nancy: Dependency vulnerabilities

• WebApp Run Time Testing: language neutral

  • Selenium: many language options
  • Puppeteer: (NOT Puppet)
  • LogRocket: Web client js logging and usage analytics.
  • Load Testing
    • LightHouse Build time score generation with Lighthouse CLI for Node. Consider as “code quality” tool.
    • Apache JMeter
    • Gatling
    • The Grinder
  • A11y (Accessibility)
    • LightHouse Build time score generation with Lighthouse CLI for Node. Consider as “code quality” tool.

CI/CD Tools

• Leverages local build scripting and runtime testing tools, adds integration, optional deployment and delivery

  • Jenkins
    • SonarQube Static code analysis
  • GitHub Actions
  • UrbanCode
    • UrbanCode Build
    • UrbanCode Deploy
  • CloudBees (CodeShip)
  • TeamCity: JetBrains
  • Travis CI
  • Circle CI
  • AWS CodePipeline
    • AWS CodeBuild
    • AWS CodeDeploy
  • Azure DevOps
    • Azure Test Plans
    • Azure Pipelines
  • Security Scan
    • OWASP (Open Web App Security Project)
    • Probely (sic) Security Scanner
    • Sonatype Integrations

Operations

• SysOps practices and tooling overlap with DevOps.
• For more details, see my SysOps and/or Containers pages.

Monitor

• Remote monitoring should include anonymous monitoring agents in webapps.
• Promethius
• Grafana

Continuous Feedback

New tickets or new issues on old tickets feed back into the Plan phase.

The DevOps journey “goes on and on and on and on”; don’t stop believin’.