This python library is a set of tools to manage AWS Doc SDK Example metadata. It is used by the AWS Doc SDK Examples team, as well as tributary sources of example snippets.

-tools:

• Validates example metadata.
• Provides an API to program against example metadata.
• Hydrates additional derived data not explicitly written by engineers into example metadata.

The check-in tests are run whenever a pull request is submitted or changed. They can be included in a Github Action with a job like this:

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - name: checkout repo content
        uses: actions/checkout@v4
      - name: validate metadata
        uses: awsdocs/aws-doc-sdk-examples-tools@main

The check-in tests walk the full repository and scan code files to look for the following issues.

• Disallow a list of specified words.
• Disallow any 20- or 40- character strings that fit a specified regex profile that indicates they might be secret access keys. Allow strings that fit the regex profile if they are in the allow list.
• Disallow file names that contain 20- or 40- character strings that fit the same regex profile, unless the filename is in the allow list.
• Verify that snippet-start and snippet-end tags are in matched pairs. You are not required to include these tags, but if you do they must be in pairs.
• Ensures any snippet_file in metadata excerpts are present in the repo.

A count of errors found is returned. When CI receives a non-zero return code, it treats the checks as failed and displays a message in the pull request.

The above configuration tracks the main branch directly. To follow more stable releases, use the most recent release tag in the github action.

uses: awsdocs/aws-doc-sdk-examples-tools@2024-08-26-A

python3.8 -m venv .venv
# With a python 3.8 venv in .venv
source .venv/bin/activate # Adjust for windows as necessary
python -m pip install -r requirements.txt
python -m pip install -e .
python -m mypy aws_doc_sdk_examples_tools
python -m pytest -vv
python -m black --check

Some validation options can be extended by creating .doc_gen/validation.yaml.

• allow_list: The 40-character check is very sensitive. To allow certain patterns, add them as a string to the allow_list key, which will be loaded as a set of strings to allow.
• sample_files: Sample files are only allowed with certain names. To allow additional sample files, add their file name (with extension, but not path) to this list.

There are two stages, testing and deployment.

1. Create a testing branch from aws-doc-sdk-examples@main.
2. Find the most recent commit SHA in aws-doc-sdk-examples-tools/commits/main.
3. Update your testing branch: With your commit SHA (format: org/repo@hash, e.g. awsdocs/aws-doc-sdk-examples-tools@e7c283e916e8efc9113277e2f38c8fa855a79d0a), update the following files:
   • In .github/workflows/validate-doc-metadata.yml, replace the current tag with the SHA.
   • Add only the commit SHA to the allow_list field in .doc_gen/validation.yaml.
4. Open a Draft PR to main branch: Do not publish for review. Wait for checks/tests to pass on the PR.

1. Update the -tools version: Once the tests pass, update the setup.py version and create a tag in the -tools repository at the same SHA you identified earlier.

   • Determine the next semver number as appropriate for the changes in this release. e.g. 1.1.4

   • Create a local version identifier, with the format YYYY-MM-DD-A, where YYYY-MM-DD represents release date, and -A is used for the first release of the day (followed by -B, -C, etc., for subsequent same-day releases). e.g. 2024-08-24-A

     • Here is a command line script to generate the local identifier, tested on Mac:

     TAG_NAME=$(date +%Y-%m-%d)-A && \
       SHA=$(git rev-parse HEAD) && \
       git tag -a "$TAG_NAME" "$SHA" -m "Release $TAG_NAME" && \
       git push origin "$TAG_NAME"
     

   • The new version will be {NEXT_SEMVER}+{LOCAL_VERSION}, e.g. 1.1.4+2024-08-24-A

     • Update setup.py with this version.

   • Create a tag at the SHA from the testing phase, using the local version identifier.

2. Update your testing PR branch: Remove SHA and add tag to validate-doc-metadata.yml

   • NOTE: Remove the SHA from .doc_gen/validation.yaml
   • This is easily accomplished in the UI.

3. Create a release: Use the automated "Create release from tag" button to create a new release with the new tag.

4. Perform internal update process.

See CONTRIBUTING for more information.

This project is licensed under the Apache-2.0 License.