2019 twenty four merry days of Perl Feed

Core Strength

Module::CoreList - 2019-12-18

One of the things to consider in the brave new world of Perl Lambda functions we've been exploring together over the last few days, is dependance cost. Every individual web endpoint is effectively its own bundle of dependencies. If we're not careful, the dependencies can add up quickly - for example, DateTime and its required dependencies alone adds 30MB+ of source code and support files. This is a problem when we've got a platform enforced hard limit of 50MB of decompressed files in our Lambda sourcecode zipfile - we either need to start splitting things out into layers, or slimming down.

One thing we should not overlook is that Perl already comes with a collection of awesome modules for doing the core tings (hence the name - the core distribution.) We should use more of those when we can! But how do we know if chosen module is available?

Module::CoreList is a CPAN module designed to solve this problem. It keeps detailed information on what core modules were included with the main Perl distribution for each release.

It's accessible on the web from http://corelist.rpee.be/, via the module interface, or most simply, via a command line tool.

Streching our Core Muscles

The command line tool has a bunch of really handy options - let's take tour and see what we can work out.

How about when a perl module was first released with perl?

    shell$ corelist List::Util
    Data for 2019-11-20
    List::Util was first released with perl v5.7.3

What was the date that went live?

    shell$ corelist -r 5.7.3
    Perl v5.7.3 was released on 2002-03-05

According to the List::Util's documentation the any function has been available since version 1.33. What version of perl was that released with?

    shell$ corelist -a List::Util
    Data for 2019-11-20
    List::Util was first released with perl v5.7.3
    v5.7.3     1.06_00   
    v5.8.0     1.07_00   
    v5.8.1     1.13      
    v5.8.2     1.13      
    v5.8.3     1.13      
    v5.8.4     1.13      
    v5.8.5     1.14      
    v5.8.6     1.14      
    v5.8.7     1.14      
    v5.8.8     1.18      
    v5.8.9     1.19      
    v5.9.0     1.13      
    v5.9.1     1.13      
    v5.9.2     1.14      
    v5.9.3     1.18      
    v5.9.4     1.18      
    v5.9.5     1.19      
    v5.10.0    1.19      
    v5.10.1    1.21      
    v5.11.0    1.21      
    v5.11.1    1.21      
    v5.11.2    1.22      
    v5.11.3    1.22      
    v5.11.4    1.22      
    v5.11.5    1.22      
    v5.12.0    1.22      
    v5.12.1    1.22      
    v5.12.2    1.22      
    v5.12.3    1.22      
    v5.12.4    1.22      
    v5.12.5    1.22      
    v5.13.0    1.22      
    v5.13.1    1.23      
    v5.13.2    1.23      
    v5.13.3    1.23      
    v5.13.4    1.23      
    v5.13.5    1.23      
    v5.13.6    1.23      
    v5.13.7    1.23      
    v5.13.8    1.23      
    v5.13.9    1.23      
    v5.13.10   1.23      
    v5.13.11   1.23      
    v5.14.0    1.23      
    v5.14.1    1.23      
    v5.14.2    1.23      
    v5.14.3    1.23      
    v5.14.4    1.23      
    v5.15.0    1.23      
    v5.15.1    1.23      
    v5.15.2    1.23      
    v5.15.3    1.23      
    v5.15.4    1.23      
    v5.15.5    1.23      
    v5.15.6    1.23      
    v5.15.7    1.23      
    v5.15.8    1.23      
    v5.15.9    1.23      
    v5.16.0    1.23      
    v5.16.1    1.25      
    v5.16.2    1.25      
    v5.16.3    1.25      
    v5.17.0    1.23      
    v5.17.1    1.25      
    v5.17.2    1.25      
    v5.17.3    1.25      
    v5.17.4    1.25      
    v5.17.5    1.25      
    v5.17.6    1.25      
    v5.17.7    1.26      
    v5.17.8    1.27      
    v5.17.9    1.27      
    v5.17.10   1.27      
    v5.17.11   1.27      
    v5.18.0    1.27      
    v5.18.1    1.27      
    v5.18.2    1.27      
    v5.18.3    1.27      
    v5.18.4    1.27      
    v5.19.0    1.27      
    v5.19.1    1.27      
    v5.19.2    1.27      
    v5.19.3    1.31      
    v5.19.4    1.32      
    v5.19.5    1.35      
    v5.19.6    1.35      
    v5.19.7    1.35      
    v5.19.8    1.36      
    v5.19.9    1.38      
    v5.19.10   1.38      
    v5.19.11   1.38      
    v5.20.0    1.38      
    v5.20.1    1.38      
    v5.20.2    1.38      
    v5.20.3    1.38      
    v5.21.0    1.38      
    v5.21.1    1.39      
    v5.21.2    1.39      
    v5.21.3    1.39      
    v5.21.4    1.41      
    v5.21.5    1.41      
    v5.21.6    1.41      
    v5.21.7    1.41      
    v5.21.8    1.41      
    v5.21.9    1.41      
    v5.21.10   1.41      
    v5.21.11   1.41      
    v5.22.0    1.41      
    v5.22.1    1.41      
    v5.22.2    1.41      
    v5.22.3    1.41      
    v5.22.4    1.41      
    v5.23.0    1.42_01   
    v5.23.1    1.42_01   
    v5.23.2    1.42_01   
    v5.23.3    1.42_01   
    v5.23.4    1.42_01   
    v5.23.5    1.42_01   
    v5.23.6    1.42_01   
    v5.23.7    1.42_01   
    v5.23.8    1.42_01   
    v5.23.9    1.42_02   
    v5.24.0    1.42_02   
    v5.24.1    1.42_02   
    v5.24.2    1.42_02   
    v5.24.3    1.42_02   
    v5.24.4    1.42_02   
    v5.25.0    1.42_02   
    v5.25.1    1.45_01   
    v5.25.2    1.45_01   
    v5.25.3    1.45_01   
    v5.25.4    1.45_01   
    v5.25.5    1.45_01   
    v5.25.6    1.46      
    v5.25.7    1.46_02   
    v5.25.8    1.46_02   
    v5.25.9    1.46_02   
    v5.25.10   1.46_02   
    v5.25.11   1.46_02   
    v5.25.12   1.46_02   
    v5.26.0    1.46_02   
    v5.26.1    1.46_02   
    v5.26.2    1.46_02   
    v5.26.3    1.46_02   
    v5.27.0    1.46_02   
    v5.27.1    1.46_02   
    v5.27.2    1.48      
    v5.27.3    1.48      
    v5.27.4    1.48      
    v5.27.5    1.49      
    v5.27.6    1.49      
    v5.27.7    1.49      
    v5.27.8    1.49      
    v5.27.9    1.49      
    v5.27.10   1.50      
    v5.27.11   1.50      
    v5.28.0    1.50      
    v5.28.1    1.50      
    v5.28.2    1.50      
    v5.29.0    1.50      
    v5.29.1    1.50      
    v5.29.2    1.50      
    v5.29.3    1.50      
    v5.29.4    1.50      
    v5.29.5    1.50      
    v5.29.6    1.50      
    v5.29.7    1.50      
    v5.29.8    1.50      
    v5.29.9    1.50      
    v5.29.10   1.50      
    v5.30.0    1.50      
    v5.30.1    1.50      
    v5.31.0    1.50      
    v5.31.1    1.50      
    v5.31.2    1.50      
    v5.31.3    1.50      
    v5.31.4    1.52      
    v5.31.5    1.52      
    v5.31.6    1.53      

Or, more sucinctly:

    corelist -a List::Util | perl -ne '/^\s+(\S+)\s+(\S+)/ && $2 >= 1.33 && print && exit'
      v5.19.5    1.35      

If I find a bug in the module do I have to install a new version of perl to get an update, or can I just download an updated version of the module from the CPAN? And where can I report that bug to?

    corelist -u List::Util
    Data for 2019-11-20
    List::Util was first released with perl v5.7.3
    upstream: cpan
    bug tracker: https://rt.cpan.org/Public/Dist/Display.html?Name=Scalar-List-Utils

(I can get a new version from the CPAN, and that's the bug tracker URL)

    corelist -u PerlIO
    Data for 2019-11-20
    PerlIO was first released with perl v5.7.3
    upstream: undef
    bug tracker: unknown

(PerlIO is shipped with perl, and I report bugs the same way I do with the core language itself, via perlbug)

What do we see if we search for a module that doesn't exist in core (even if we think it should be)

    shell$ corelist DateTime

    Data for 2019-11-20
    DateTime was not in CORE (or so I think)

This, of course, doesn't mean that DateTime might not come preinstalled on your system (on macOS for example it's distributed with the operating system.) Just it's not in the main core distribution!

What about something that was added to the core distribution but then removed again?

    $ corelist CGI
    Data for 2019-11-20
    CGI was first released with perl 5.004, deprecated (will be CPAN-only)
    in v5.19.7 and removed from v5.21.0

What about a core language level feature

    shell$ corelist -f say
    Data for 2019-11-20
    feature "say" was first released with the perl v5.9.5 feature bundle

(this means you can use feature "say" on any perl after v5.9.5)

You can even find out what were the module changes between Perl 5.28 and 5.30 for example:

    $ corelist --diff v5.28.0 v5.30.0
    App::Cpan                                 1.67      1.672
    Archive::Tar                              2.30       2.32
    Archive::Tar::Constant                    2.30       2.32
    Archive::Tar::File                        2.30       2.32
    B                                         1.74       1.76
    B::Concise                               1.003      1.004
    B::Debug                                  1.26   (absent)
    B::Deparse                                1.48       1.49
    B::Op_private                         5.028000   5.030000
    B::Terse                                  1.08       1.09
    CPAN                                      2.20       2.22
    CPAN::Distribution                        2.19       2.22
    CPAN::Mirrors                             2.12       2.21
    CPAN::Plugin                              0.96       0.97
    CPAN::Plugin::Specfile                    0.01       0.02
    CPAN::Shell                             5.5007     5.5008
    Compress::Raw::Bzip2                     2.074      2.084
    Compress::Raw::Zlib                      2.076      2.084
    Compress::Zlib                           2.074      2.084
    Config                                   5.028       5.03
    Config::Extensions                        0.02       0.03
    Config::Perl::V                           0.29       0.32
    Cwd                                       3.74       3.78
    DB_File                                  1.840      1.843
    Data::Dumper                             2.170      2.174
    Devel::PPPort                             3.40       3.52
    Devel::Peek                               1.27       1.28
    Digest::SHA                               6.01       6.02
    Encode                                    2.97       3.01
    Encode::Unicode                           2.17       2.18
    Errno                                     1.29       1.30
    ExtUtils::CBuilder                    0.280230   0.280231
    ExtUtils::CBuilder::Base              0.280230   0.280231
    ExtUtils::CBuilder::Platform::Unix    0.280230   0.280231
    ExtUtils::CBuilder::Platform::VMS     0.280230   0.280231
    ExtUtils::CBuilder::Platform::Windows   0.280230   0.280231
    ExtUtils::CBuilder::Platform::Windows::BCC   0.280230   0.280231
    ExtUtils::CBuilder::Platform::Windows::GCC   0.280230   0.280231
    ExtUtils::CBuilder::Platform::Windows::MSVC   0.280230   0.280231
    ExtUtils::CBuilder::Platform::aix     0.280230   0.280231
    ExtUtils::CBuilder::Platform::android   0.280230   0.280231
    ExtUtils::CBuilder::Platform::cygwin   0.280230   0.280231
    ExtUtils::CBuilder::Platform::darwin   0.280230   0.280231
    ExtUtils::CBuilder::Platform::dec_osf   0.280230   0.280231
    ExtUtils::CBuilder::Platform::os2     0.280230   0.280231
    ExtUtils::Manifest                        1.70       1.72
    ExtUtils::Miniperl                        1.08       1.09
    ExtUtils::ParseXS                         3.39       3.40
    ExtUtils::ParseXS::Constants              3.39       3.40
    ExtUtils::ParseXS::CountLines             3.39       3.40
    ExtUtils::ParseXS::Eval                   3.39       3.40
    ExtUtils::ParseXS::Utilities              3.39       3.40
    File::Copy                                2.33       2.34
    File::Find                                1.34       1.36
    File::Glob                                1.31       1.32
    File::GlobMapper                         1.000      1.001
    File::Path                                2.15       2.16
    File::Spec                                3.74       3.78
    File::Spec::AmigaOS                       3.74       3.78
    File::Spec::Cygwin                        3.74       3.78
    File::Spec::Epoc                          3.74       3.78
    File::Spec::Functions                     3.74       3.78
    File::Spec::Mac                           3.74       3.78
    File::Spec::OS2                           3.74       3.78
    File::Spec::Unix                          3.74       3.78
    File::Spec::VMS                           3.74       3.78
    File::Spec::Win32                         3.74       3.78
    File::Temp                              0.2304     0.2309
    Filter::Util::Call                        1.58       1.59
    GDBM_File                                 1.17       1.18
    HTTP::Tiny                               0.070      0.076
    I18N::Langinfo                            0.17       0.18
    IO                                        1.39       1.40
    IO::Compress::Adapter::Bzip2             2.074      2.084
    IO::Compress::Adapter::Deflate           2.074      2.084
    IO::Compress::Adapter::Identity          2.074      2.084
    IO::Compress::Base                       2.074      2.084
    IO::Compress::Base::Common               2.074      2.084
    IO::Compress::Bzip2                      2.074      2.084
    IO::Compress::Deflate                    2.074      2.084
    IO::Compress::Gzip                       2.074      2.084
    IO::Compress::Gzip::Constants            2.074      2.084
    IO::Compress::RawDeflate                 2.074      2.084
    IO::Compress::Zip                        2.074      2.084
    IO::Compress::Zip::Constants             2.074      2.084
    IO::Compress::Zlib::Constants            2.074      2.084
    IO::Compress::Zlib::Extra                2.074      2.084
    IO::Dir                                   1.39       1.40
    IO::File                                  1.39       1.40
    IO::Handle                                1.39       1.40
    IO::Pipe                                  1.39       1.40
    IO::Poll                                  1.39       1.40
    IO::Seekable                              1.39       1.40
    IO::Select                                1.39       1.40
    IO::Socket                                1.39       1.40
    IO::Socket::INET                          1.39       1.40
    IO::Socket::UNIX                          1.39       1.40
    IO::Uncompress::Adapter::Bunzip2         2.074      2.084
    IO::Uncompress::Adapter::Identity        2.074      2.084
    IO::Uncompress::Adapter::Inflate         2.074      2.084
    IO::Uncompress::AnyInflate               2.074      2.084
    IO::Uncompress::AnyUncompress            2.074      2.084
    IO::Uncompress::Base                     2.074      2.084
    IO::Uncompress::Bunzip2                  2.074      2.084
    IO::Uncompress::Gunzip                   2.074      2.084
    IO::Uncompress::Inflate                  2.074      2.084
    IO::Uncompress::RawInflate               2.074      2.084
    IO::Uncompress::Unzip                    2.074      2.084
    IPC::Cmd                                  1.00       1.02
    JSON::PP                               2.97001       4.02
    JSON::PP::Boolean                      2.97001       4.02
    Locale::Codes                             3.56   (absent)
    Locale::Codes::Constants                  3.56   (absent)
    Locale::Codes::Country                    3.56   (absent)
    Locale::Codes::Country_Codes              3.56   (absent)
    Locale::Codes::Country_Retired            3.56   (absent)
    Locale::Codes::Currency                   3.56   (absent)
    Locale::Codes::Currency_Codes             3.56   (absent)
    Locale::Codes::Currency_Retired           3.56   (absent)
    Locale::Codes::LangExt                    3.56   (absent)
    Locale::Codes::LangExt_Codes              3.56   (absent)
    Locale::Codes::LangExt_Retired            3.56   (absent)
    Locale::Codes::LangFam                    3.56   (absent)
    Locale::Codes::LangFam_Codes              3.56   (absent)
    Locale::Codes::LangFam_Retired            3.56   (absent)
    Locale::Codes::LangVar                    3.56   (absent)
    Locale::Codes::LangVar_Codes              3.56   (absent)
    Locale::Codes::LangVar_Retired            3.56   (absent)
    Locale::Codes::Language                   3.56   (absent)
    Locale::Codes::Language_Codes             3.56   (absent)
    Locale::Codes::Language_Retired           3.56   (absent)
    Locale::Codes::Script                     3.56   (absent)
    Locale::Codes::Script_Codes               3.56   (absent)
    Locale::Codes::Script_Retired             3.56   (absent)
    Locale::Country                           3.56   (absent)
    Locale::Currency                          3.56   (absent)
    Locale::Language                          3.56   (absent)
    Locale::Script                            3.56   (absent)
    Math::BigFloat                        1.999811   1.999816
    Math::BigFloat::Trace                     0.49       0.51
    Math::BigInt                          1.999811   1.999816
    Math::BigInt::Calc                    1.999811   1.999816
    Math::BigInt::CalcEmu                 1.999811   (absent)
    Math::BigInt::FastCalc                  0.5006     0.5008
    Math::BigInt::Lib                     1.999811   1.999816
    Math::BigInt::Trace                       0.49       0.51
    Math::BigRat                            0.2613     0.2614
    Module::CoreList                    5.20180622 5.20190522
    Module::CoreList::Utils             5.20180622 5.20190522
    Module::Load                              0.32       0.34
    Module::Metadata                      1.000033   1.000036
    NDBM_File                                 1.14       1.15
    Net::Ping                                 2.62       2.71
    ODBM_File                                 1.15       1.16
    POSIX                                     1.84       1.88
    PerlIO::encoding                          0.26       0.27
    PerlIO::scalar                            0.29       0.30
    Pod::Man                                  4.10       4.11
    Pod::ParseLink                            4.10       4.11
    Pod::Text                                 4.10       4.11
    Pod::Text::Color                          4.10       4.11
    Pod::Text::Overstrike                     4.10       4.11
    Pod::Text::Termcap                        4.10       4.11
    SDBM_File                                 1.14       1.15
    Storable                                  3.08       3.15
    Storable::Limit                        (undef)   (absent)
    Test2                                 1.302133   1.302162
    Test2::API                            1.302133   1.302162
    Test2::API::Breakage                  1.302133   1.302162
    Test2::API::Context                   1.302133   1.302162
    Test2::API::Instance                  1.302133   1.302162
    Test2::API::Stack                     1.302133   1.302162
    Test2::Event                          1.302133   1.302162
    Test2::Event::Bail                    1.302133   1.302162
    Test2::Event::Diag                    1.302133   1.302162
    Test2::Event::Encoding                1.302133   1.302162
    Test2::Event::Exception               1.302133   1.302162
    Test2::Event::Fail                    1.302133   1.302162
    Test2::Event::Generic                 1.302133   1.302162
    Test2::Event::Note                    1.302133   1.302162
    Test2::Event::Ok                      1.302133   1.302162
    Test2::Event::Pass                    1.302133   1.302162
    Test2::Event::Plan                    1.302133   1.302162
    Test2::Event::Skip                    1.302133   1.302162
    Test2::Event::Subtest                 1.302133   1.302162
    Test2::Event::TAP::Version            1.302133   1.302162
    Test2::Event::V2                      1.302133   1.302162
    Test2::Event::Waiting                 1.302133   1.302162
    Test2::EventFacet                     1.302133   1.302162
    Test2::EventFacet::About              1.302133   1.302162
    Test2::EventFacet::Amnesty            1.302133   1.302162
    Test2::EventFacet::Assert             1.302133   1.302162
    Test2::EventFacet::Control            1.302133   1.302162
    Test2::EventFacet::Error              1.302133   1.302162
    Test2::EventFacet::Hub                1.302133   1.302162
    Test2::EventFacet::Info               1.302133   1.302162
    Test2::EventFacet::Info::Table        (absent)    (undef)
    Test2::EventFacet::Meta               1.302133   1.302162
    Test2::EventFacet::Parent             1.302133   1.302162
    Test2::EventFacet::Plan               1.302133   1.302162
    Test2::EventFacet::Render             1.302133   1.302162
    Test2::EventFacet::Trace              1.302133   1.302162
    Test2::Formatter                      1.302133   1.302162
    Test2::Formatter::TAP                 1.302133   1.302162
    Test2::Hub                            1.302133   1.302162
    Test2::Hub::Interceptor               1.302133   1.302162
    Test2::Hub::Interceptor::Terminator   1.302133   1.302162
    Test2::Hub::Subtest                   1.302133   1.302162
    Test2::IPC                            1.302133   1.302162
    Test2::IPC::Driver                    1.302133   1.302162
    Test2::IPC::Driver::Files             1.302133   1.302162
    Test2::Tools::Tiny                    1.302133   1.302162
    Test2::Util                           1.302133   1.302162
    Test2::Util::ExternalMeta             1.302133   1.302162
    Test2::Util::Facets2Legacy            1.302133   1.302162
    Test2::Util::HashBase                 1.302133   1.302162
    Test2::Util::Trace                    1.302133   1.302162
    Test::Builder                         1.302133   1.302162
    Test::Builder::Formatter              1.302133   1.302162
    Test::Builder::Module                 1.302133   1.302162
    Test::Builder::Tester                 1.302133   1.302162
    Test::Builder::Tester::Color          1.302133   1.302162
    Test::Builder::TodoDiag               1.302133   1.302162
    Test::More                            1.302133   1.302162
    Test::Simple                          1.302133   1.302162
    Test::Tester                          1.302133   1.302162
    Test::Tester::Capture                 1.302133   1.302162
    Test::Tester::CaptureRunner           1.302133   1.302162
    Test::Tester::Delegate                1.302133   1.302162
    Test::use::ok                         1.302133   1.302162
    Thread::Queue                             3.12       3.13
    Time::HiRes                             1.9759     1.9760
    Time::Local                               1.25       1.28
    Time::Piece                             1.3204       1.33
    Time::Seconds                           1.3204       1.33
    Unicode                                 10.0.0     12.1.0
    Unicode::Collate                          1.25       1.27
    Unicode::Collate::CJK::Big5               1.25       1.27
    Unicode::Collate::CJK::GB2312             1.25       1.27
    Unicode::Collate::CJK::JISX0208           1.25       1.27
    Unicode::Collate::CJK::Korean             1.25       1.27
    Unicode::Collate::CJK::Pinyin             1.25       1.27
    Unicode::Collate::CJK::Stroke             1.25       1.27
    Unicode::Collate::CJK::Zhuyin             1.25       1.27
    Unicode::Collate::Locale                  1.25       1.27
    Unicode::UCD                              0.70       0.72
    User::grent                               1.02       1.03
    XS::APItest                               0.98       1.00
    XS::Typemap                               0.16       0.17
    arybase                                   0.15   (absent)
    bigint                                    0.49       0.51
    bignum                                    0.49       0.51
    bigrat                                    0.49       0.51
    bytes                                     1.06       1.07
    deprecate                                 0.03       0.04
    experimental                             0.019      0.020
    feature                                   1.52       1.54
    lib                                       0.64       0.65
    ok                                    1.302133   1.302162
    parent                                   0.236      0.237
    perlfaq                               5.021011 5.20190126
    re                                        0.36       0.37
    sigtrap                                   1.08       1.09
    threads::shared                           1.58       1.60
    utf8                                      1.21       1.22
    vars                                      1.04       1.05
    version                                 0.9923     0.9924
    version::regex                          0.9923     0.9924
    warnings                                  1.42       1.44

Beyond a Core Workout

One thing to consider however is that core modules aren't your own source of modules on the enviroment.

Consider running a Lambda function and using Try::Tiny. Not a core module:

    $ corelist Try::Tiny
    Data for 2019-11-20
    Try::Tiny was not in CORE (or so I think)

But you can still use it in an AWS::Lambda powered function without having to supply it yourself in the zipfile! How is this possible? Well, it's because AWS::Lambda itself uses the module, and therefore it's in the default layer that your Lambda function uses.

What you need to do is look at the dependance graph for AWS::Lambda so you can see not only the direct dependencies of AWS::Lambda but its dependencies' dependencies and so on. You can probably use all of these too!

You can literally see a graph for each module's dependancies by visiting https://cpandeps.grinnz.com/. For example, here's the graph for AWS::Lambda.

Similarly, we noted that DateTime is available on macOS Catalina. Therefore, any runtime dependencies it requires must also be installed, correct? Well, it's not quite that easy. You'd expect the excellent Params::ValidationCompiler to be available as it's a requirement of the DateTime 1.51 module that's on the CPAN. But macOS only bundles version 1.06, which doesn't require Params::ValidationCompiler, so it's not there. You have to sweat the details!

Keep Working It

Of course, if you need something from the CPAN, you should go ahead and grab it. But, sometimes it's good to run slim...

Gravatar Image This article contributed by: Mark Fowler <mark@twoshortplanks.com>