A warning during build

I tried to build the module locally and run its tests locally, but I encountered some issues:

First I noticed that there are some warning during the build emitted by one of the dependencies. As it turns out upgrading the dependency solved this issue, but the latest package of the dependency also had a minor issue. The version numbers in the different files were slightly confusing. So I reported that too.

Apparently that was already fixed, it was just not released yet.

An error - missing files

The next thing I encountered was that some files were missing. That made the tests fail. The author first thought that the issue was somehow cause by me mixing versions, but soon found that the files were not added to git.

This is one of the reasons to have CI.

This can happen to any one of us.

Maybe not to you, but someone else on your team or to a contributor to your project

Should the tip of the main branch be usable?

The author made an interesting comment on the issue about the missing files.

FYI many programmers make a point of the tip of their main branch being something usable. I am NOT one of them, so pulling from my repo, instead of a distribution, is risky.

Here is what I think about it:

I would say it is a good practice to ensure all the test are always passing on the tip of the main branch. Otherwise one would risk starting to think that "some tests always fail" which could lead to some test failing that *should not fail* go unnoticed for a long period of time.

Also it upsets the CI and we don't want to do that.

Actually, one of the reasons of having a CI to have something look over our shoulders and make sure that all the tests are always passing on a clean environment.

And then comes the point, if all the tests are passing, doesn't that mean that the version is usable?

I am looking forward the continuation of this discussion.

Pull-Request

Once those issues were fixed adding the GitHub Action configuration file and converting the commands from the Travis-CI config file to GitHub Actions was not difficult. At one point the author commented that he wants the code to run on Windows and macOS as well, but it is difficult as he does not have access to those operating systems. I figured configuring GitHub Actions to run on those OS-es as well will help him with that task. I was also expecting the test or even maybe the compilation to fail on at lest one of them. So I was a bit surprised that everything worked.

Well, I still managed to make a few mistakes and thus had to try it several time, but after a while I got it right and sent the pull-request. To my surprise the author closed it as he is planning to move the repo to a GitHub organization.

It is unclear to me why that stops him from accepting the PR so I asked him on the PR.

GitHub Actions

In any case, I included the configuration files here. This time, I went a bit further than in the previous cases of my recent GitHub Actions pull-requests. I also went beyond what was in the Travis-CI configuration file.

This time I create two configuration file.

In the first one we use multiple version of perl in Docker containers.

examples/marpa-r2/ci.yml

name: CI

on:
  push:
  pull_request:
  workflow_dispatch:
  schedule:
    - cron: '42 5 * * *'

jobs:
  test:
    strategy:
      fail-fast: false
      matrix:
        perl: [ '5.10', '5.20', '5.30', '5.36', '5.36-threaded' ]
        # See options here: https://hub.docker.com/_/perl/

    runs-on: ubuntu-latest
    name: OS Perl ${{matrix.perl}}
    container: perl:${{matrix.perl}}

    steps:
    - uses: actions/checkout@v3

    - name: Show Perl Version
      run: |
        perl -v

    - name: Install Modules
      run: |
        cpanm Config::AutoConf # optional dependency
        cd cpan
        cpanm --installdeps --quiet --notest .

    - name: Show Errors on Ubuntu
      if:  ${{ failure() && startsWith( matrix.runner, 'ubuntu-')}}
      run: |
         cat /home/runner/.cpanm/work/*/build.log

    - name: build Marpa and execute tests
      env:
        AUTHOR_TESTING: 1
        RELEASE_TESTING: 1
      run: |
        set -x
        (cd cpan/xs/ && make)  # generate necessary files

        cd cpan/
        perl Build.PL
        ./Build
        ./Build test
        ./Build distmeta
        ./Build disttest
        MARPA_USE_PERL_AUTOCONF=1 ./Build disttest
        ./Build dist

In the second one we use the 3 different operating systems: Windows, Linux and macOS.

examples/marpa-r2/native.yml

name: Platforms

on:
  push:
  pull_request:
  workflow_dispatch:
#  schedule:
#    - cron: '42 5 * * *'

jobs:
  platform-test:
    strategy:
      fail-fast: false
      matrix:
        runner: [ubuntu-latest, macos-latest, windows-latest]
        perl: [ '5.30' ]
        exclude:
          - runner: windows-latest
            perl: '5.36'

    runs-on: ${{matrix.runner}}
    name: OS ${{matrix.runner}} Perl ${{matrix.perl}}

    steps:
    - uses: actions/checkout@v3

    - name: Set up perl
      uses: shogo82148/actions-setup-perl@v1
      with:
          perl-version: ${{ matrix.perl }}
          distribution: ${{ ( matrix.runner == 'windows-latest' && 'strawberry' ) || 'default' }}

    - name: Show Perl Version
      run: |
        perl -v

    - name: Install Modules
      run: |
        cpanm Config::AutoConf # optional dependency
        cd cpan
        cpanm --installdeps --quiet --notest .

    - name: Show Errors on Windows
      if:  ${{ failure() && matrix.runner == 'windows-latest' }}
      run: |
         ls -l C:/Users/
         ls -l C:/Users/RUNNER~1/
         cat C:/Users/runneradmin/.cpanm/work/*/build.log

    - name: Show Errors on Ubuntu
      if:  ${{ failure() && matrix.runner == 'ubuntu-latest' }}
      run: |
         cat /home/runner/.cpanm/work/*/build.log

    - name: Show Errors on OSX
      if:  ${{ failure() && matrix.runner == 'macos-latest' }}
      run: |
         cat  /Users/runner/.cpanm/work/*/build.log

    - name: Make - generate necessary files
      run: |
        cd cpan
        cd xs
        make

    - name: Run tests
      env:
        AUTHOR_TESTING: 1
        RELEASE_TESTING: 1
      run: |
        cd cpan
        perl Build.PL
        perl Build
        perl Build test
        perl Build distmeta
        perl Build disttest

    - name: Run tests with autoconf
      env:
        MARPA_USE_PERL_AUTOCONF: 1
      run: |
        cd cpan
        perl Build disttest

    #- name: Build dist
    #  env:
    #    perl Build dist

Conclusion

You don't have to have a CI if you remember always running your tests. You don't even need to have access to Windows and macOS to make your code work there if you are a really good programmer as Jeffrey Kegler, the author of Marpa. And even then you might forget to add some of the files to git.

However, most of us aren't as focused on the details and most of would not be able to build a project like Marpa. For sure I know I wouldn't.

So we need the hand-holding and the discipline a CI can provide.