In software with plugin capabilities, typically in interpreted languages, it is possible that plugin have dependencies included, that may collide with dependencies included in core product, or in other plugins. It is not a problem when the same version is used, but in turns into a problem when different versions that are incompatible to each other are shipped. The end user may face bugs. Developers speak of the Dependency Hell.

In Nextcloud these plugins are the various apps, coming from various sources.

Example:

• The SharePoint backend indirectly (dependency of a dependency) ships firebase/php-jwt in version 6.10.0.
• The SAML/SSO backend ships firebase/php-jwt in version 6.8.1
• By loading their classes dynamically, it is unclear which will be loaded
• … but both cannot be loaded twice – there cannot be the same classes twice
• I do not know whether these versions are compatible now, but if they were…
• … it might look very different with the next update

While managing dependencies is not exactly a new thing, the existing solutions in PHP land are not that manifold or feature rich. Some apps use mozart for that purpose. mozart was oriented on WordPress plugins, though, but it is also not in active development anymore.

The other available tool is php-scoper and this is what the blog post is about. From my example, one direct dependency pulled in the colliding one in a new version. To ensure that there will be no shenanigans caused, I had to isolate my apps dependencies. To scope them in other words.

This is achieved by moving the dependencies into my apps namespace. By doing so, the dependencies will not be loaded anymore with their original namespaces class name, but the app-specific one. So, they can be loaded multiple times and not interfere with each other.

As we did not have prior art with php-scoper, I could explore my way into it.

Objectives

First, an idea and plan will be the foundation that paints the picture of the desired outcome. And in order to not totally reinvent the wheel, I looked at a battle-tested app, Nextcloud Talk, which is using mozart, and does some very sensible things altogether. I formulated my objectives.

1. Targeted Namespace: \OCA\{AppId}\Vendor\{OriginalNamespace}
   As hinted in the introduction, the dependencies will have to have an app-specific namespace, and this is how it shall look like.

   \OCA is the conventional and mandatory namespace prefix for apps in Nextcloud, followed by the app id with a capitalized first character.
   The next parts follow and replicate the folder structure, and by placing the dependencies in Vendor makes it obvious that these components are coming from third parties.
   Without scoping, they would just remain in the vendor directory at the apps root.

2. Development dependencies are ignored
   They do not matter as they are not shipped with in a proper release.
   In theory, you may have collisions in your development setup, and occasionally it is awkward to figure this out. But it happens once in a year with bad luck, and is not worth the effort. Probably it would make some activities even worse. So, not worth the effort.

3. lib/Vendor not committed into git
   This path, relative to the app project root, will be the parent location for the dependencies. In Nextcloud apps, lib/ is typically the source root – the starting point for \OCA{AppId}\.
   For I do not desire to have the dependencies committed to the git repository, this directory shall also not appear there. Instead the package management meta data is committed and used to fetch the code on demand.

4. Optimized autoloader with dependencies included
   For the dynamic loading of the source classes, called autoloading, the package manager composer can dump the class information for a quick lookup. The dependencies have to be there with their prefixed namespace.

5. App is able to run after composer install [--no-dev] After installing the dependencies based on the stored meta data, all the scoping steps should be applied immediately, so the app is ready without any further ado. The same is valid updating the dependencies with composer update. Any additional manual steps should be avoided for a good and consistent developer experience.

6. Successful Packaging
   Obviously, the production dependencies and autoload information should be at the destination places when creating the release archive.

Implementation

The road to get to the final result was a little bumpy for different reasons, but I will not bore with this. I outline the key steps with details where needed.

Adding php-scoper to composer

First, we make use of the bin plugin for composer as a production dependency, because we need php-scoper to also run no development-dependencies are installed – think building the release archive, or running the app without all the dev tools. So the bin plugin allows us to have php-scoper along production dependencies, but being out of the way (and easy to discard).

composer require bamarni/composer-bin-plugin
composer bin php-scoper require humbug/php-scoper=0.17.0

I had to pull php-scoper in version 0.17.0, because it is the last one that supports PHP 8.0 properly. While not a production dependency, it still runs when installing dependencies, e.g. on CI. If you do not have this restriction, a later version will suffice. For some versions of php-scoper do not state the correct PHP requirements or lack compatibility, some testing and trying may pave the way.

Configuring php-scoper

The default configuration file name is scoper.inc.php and is expected to be located in the project root directory. The namespace prefix can be defined there, as well as the directories should be arranged. Some more tricks are possible there, but I did not need them. I might have used the output-dir flag, but it is not available in 0.17.0. php-scoper will write the adjusted files to build/. For we have to move around the results anyway, it is not a big thing.

My resulting config is essentially:

<?php

declare(strict_types=1);

use Isolated\Symfony\Component\Finder\Finder;

return [
    'prefix' => 'OCA\\SharePoint\\Vendor',

    'finders' => [
        Finder::create()->files()
            ->exclude([
                'test',
                'composer',
                'bin',
            ])
            ->notName('autoload.php')
            ->in('vendor/vgrem'),
        Finder::create()->files()
            ->exclude([
                'test',
                'composer',
                'bin',
            ])
            ->notName('autoload.php')
            ->in('vendor/firebase'),
    ],
];

Some folders are (test, composer, bin) should be ignored, as well as custom autoload.phps. I was handpicking my two selected dependencies, so scoper would leave dev-deps alone. This might be not be the best approach when having a big number of dependencies.

Placing the adjusted deps to the final destination

While php-scoper is changing the sources and patching the namespace where required, it does nothing to the directory structure. This is a step I had to do, and wrote a script that adjusts the original top level directories to become compliant with PSR-4. For example:

We find:

build/vgrem/php-spo/
build/firebase/php-jwt/

and they should end up at:

lib/Vendor/Office365/
lib/Vendor/Firebase/JWT/

This is a structure has to derice from the ['autoload']['psr-4'] data out of each dependencies respective composer.json file. The structure informs about the (new) namespace as well as the sources root.

So the process is to,

1. iterate over the organization directory
2. iterate over the project directory
3. read out the psr-4 information from the composer.json
4. create the destination path (and make sure it is empty)
5. move the source to it place
6. clean up the rest

While at PHP I wrote it in PHP and stored it in the project root. It is a bit lengthy, so check it out on the code platform.

Combining it all

Now that my yak is nicely shaved I can put the last pieces together. composer post-install and post-upgrade scripts will run everything that is needed to pull the dependencies and make them ready.

"post-install-cmd": [
    "@composer bin all install --ignore-platform-reqs # unfortunately the flag is required for 8.0",
    "vendor/bin/php-scoper add-prefix --force # Scope our dependencies",
    "rm -Rf lib/Vendor && mv build lib/Vendor",
    "find lib/Vendor/ -maxdepth 1 -mindepth 1 -type d | cut -d '/' -f3 | xargs -I {} rm -Rf vendor/{} # Remove origins",
    "@php lib-vendor-organizer.php lib/Vendor/ OCA\\\\SharePoint\\\\Vendor",
    "composer dump-autoload -o"
],
"post-update-cmd": [
        …
]

Those commands are executed every time after a composer install or composer update is run – nifty. At first, php-scoper is installed through the bin plugin. Unfortunately, I have to ignore platform (PHP version) requirements to cover running on PHP 8.0 to 8.3. Try your case without that first.

The second command actually runs php-scoper, by using the configuration it finds under its default config location.

Then, we make sure that lib/Vendor is deleted and we rename the resulting build/ to this path.

The next command just cleans up the original directories in vendor/, php-scoper does not do that. We want to be sure to have no collision during our development work, so the traces of the treated dependencies should vanish (and only those).

Subsequently our script is run that adjusts the final paths. It takes two arguments, the source directory, as well as the namespace prefix. This helps avoiding hardcoding such information.

Eventually the autoload information is generated and dumped.

The update script is identical, therefore you see the ellipsis in the snippet.

Updating the owned code

What the scoper did not do for you is to adjust the namespaces in your own code files. So everywhere where classes from the old namespaces were required, they have to be prefixed with OCA{APPID}\Vendor\.

Working on a small app I did this manually in PhpStorm. With the Column Selection Mode it goes quick (one paste per file) and I see possible issues immediately. Using sed combined with (ack-)grep and/or find could be a quick alternative. Or perhaps getting deeper in the php-scoper configuration could reveal some automatic options.

Packaging

I use krankerl for packaging Nextcloud apps, and the only thing I had to adjust was to add some directories to the .nextcloudignore file, so they will not be included in the release archive. The directories added were lib-vendor-organizer.php, scoper.inc.php, vendor-bin, vendor/bin and vendor/bamarni. Otherwise the vendor directory contains the autoload configuration, which I keep at the known position.

In krankerl's configuration I was having composer install --no-dev as the sole before packaging command, and that is all that it takes (at least if you do not need frontend-code voodoo).

Push it

Do not forget to add lib/Vendor to .gitignore.

Commit everything, push it to git and let CI run over it. Fix or adjusts tests if they are not green. I did additional smoke testing. This all depends on your test coverage and possible test cases.

Different approach

While I was working on my solution, unbeknownst Dear Marcel was working on his, which resulted in a slightly different approach. Key differences are:

• Using latest php-scoper and covering only PHP 8.2 or 8.3.
• Placing the dependency directly into lib/ (the use case contained only one dependency)
• Replacing the namespace in code using sed from within the composer scripts (seems worthwhile!)
• Additional patcher in the php-scoper configuration to replace namespace in comments within the dependency codebase (also a great idea!)

Caveats and limitations

I find the series of commands in the composer script a bit hacky, and also I had to write the script to arrange the final directory structure. I would have loved if php-scoper did it itself (contribution welcome i suppose) – what i believe mozart is doing out of the box, if I did not oversee anything.

What can always be a pitfall is when dependency code is creating or instantiating classes dynamically, using string concatenation for instance. This is something to watch out for, and it might make it very tough to scope some dependencies.

Marcel encountered an issue with current versions (0.18.*) of php-scoper that lead to problems with autoloading specific files that do not contain classes, just namescaped functions. A manual require_once does solve that. A fix is already merged and will be shipped with a v1 release of php-scoper.

Recapitulate

1. Install humbug/php-scoper through the composer bin plugin, which must be a production dependency.
2. Configure php-scoper
3. Add the lib-vendor-organizer.php script
4. Configure composer post-install and post-update scripts
5. Adjust your own code
6. Test release building, run CI, do some smoke testing