Over the last two weekends I proceeded to upgrade the infrastructure behind packagist.org and getcomposer.org. There is now a lot more bandwidth and the DNS hosting was also migrated to DNSMadeEasy which should make it more stable on that front.
All in all this should result in faster composer updates which is great! I also took the chance to do a few upgrades on the config, which now includes the latest PHP/nginx versions as well as OCSP stapling directives. While looking at logs I also noticed the composer installer was not using gzip compression so I fixed that and now installing composer itself should be a bit faster too if you have the zip extension enabled.
While the whole change happened without packagist downtime, there were a few minor issues early last week with the composer homepage. Today I had to re-enable http (i.e. no-TLS) access to packagist.org as people are still relying on accessing it without the openssl extension. This is not great but the travis 5.3.3 builds are missing openssl so it's not realistic to require it, hopefully in a year or two when 5.3 is a thing of the past we can go full-TLS.
Also note that the GitHub hooks updating packages on packagist are also using http and not https so the hooks using the default domain (http://packagist.org - I sent them a PR to update it to https by default) were not properly updated in the last ~24hours. I am really sorry I missed that but there is not much I can do at this point except wait until next time a hook fires and those affected packages finally get updated.
Please get in touch on github if you spot anything else out of the ordinary (except faster updates;).
And finally a quick shout-out to our sponsors, if you haven't checked Toran Proxy out yet please do so as it allows me to pay for the hardware as well as get some time off work to focus on open source without it always being unpaid leave. Many thanks to all existing Toran customers for their support!
I tagged Composer's 1.0.0-alpha9 release yesterday and wanted to write down a more detailed update on the highlights of this release. It includes many changes as the last tag was almost one year old. You can also check the full changelog if you want more details.
Requiring packages from CLI just got easier
require command now guesses the best version constraint for the latest stable release of the package you require, so if you know you need a package but don't know what the latest version is you can just use for example
composer require monolog/monolog and it will guess and use a constraint like
~1.11. There is also a new
remove command to easily remove dependencies from the CLI without having to touch json.
Installing dependencies on the wrong environment is now possible
--ignore-platform-reqs flag for the install and update commands lets you install dependencies even if you have the wrong php version or are missing one of the required php extensions. It's not really recommended but it can be useful sometimes if you want to run composer outside a VM for example and you only have the right extensions installed in the VM where you run the code.
You now get warnings when installing abandoned packages
A few months back Packagist got this feature allowing packages to be marked as abandoned. For example this php markdown fork is not maintained anymore as the original package is now on Packagist as well. Marking it as abandoned lets people know they should ideally not rely on it in new projects, and migrate away if possible. If you install this package Composer now warns about it:
Package dflydev/markdown is abandoned, you should avoid using it. Use michelf/php-markdown instead.. This should make it easier to deprecate packages and let your users know about it.
Custom composer commands via scripts
Script handlers defined for a non-existing event are now registered as custom composer commands. So you can for example define a test handler with the command to run your test suite, and then use
composer test to call it. All the binaries installed as dependencies like phpunit in this example are made available in the PATH before the command is executed so you can call them easily without using the vendor/bin/ prefix.
Autoloading tests and related files
autoload-dev section lets you define autoload rules that only apply when your package is installed in dev mode, much like
require-dev. This is mainly useful to split off test autoload rules from the main autoload rules the users of your packages will need.
In case you missed it last week, we gained a huge performance boost by disabling garbage collection when running the Composer dependency solver. This is great news for everyone but I wanted to stress once again that you probably should not disable GC in your scripts unless you have a very good reason to do so. PHP's GC isn't flawed per se but our use case just falls out of the use cases it's been designed for. Anthony Ferrara wrote a good round-up of the issue in case you want to learn more.
A note on supporting Composer maintenance
While many of the above features have been contributed by others, reviewing pull requests and checking on new issues every day does take a lot of my time. I launched Toran Proxy about six months ago to help me get some paid time to work on and maintain Composer and Packagist. Looking at the numbers now it seems about 20% of the time spent was paid for thanks to Toran Proxy customers. I want to thank you all for supporting me and I hope more people will join in! I will try to do updates like this one more regularly to highlight new features and developments in the Composer ecosystem.
PHP 5.3 has been out of maintenance for about three months now and it seems like it's time for the community to move on at last. Drupal8 will target 5.4. Symfony announced the results of a poll about which PHP version Symfony3 should target (TL;DR: 5.5 and 5.6 are preferred). And Pascal Martin released yesterday an update on his PHP usage stats based on scanning server headers.
Pascal's number are interesting but I believe they have a bias towards older PHP versions. I would argue that people configuring their servers properly are also those that tend to keep up to date with newer versions, and part of the best practices is to avoid publishing the software versions you are using (i.e. disable expose_php in php.ini). If I am correct here that means early adopters are mis-represented in those numbers.
In any case, I do have another biased dataset to present so here it comes! I looked in the packagist.org logs of the last fifty days for
GET /packages.json which represents a composer update done by someone. Since Composer sends the PHP version it is running with in its
User-Agent header, I can use that to see which PHP versions people are using Composer with. Of course this data set is probably biased towards development machine and CI servers and as such it should also be taken with a grain of salt.
PHP usage statistics
I have two datasets, from November 2013 and today, which shows the progression of various versions. Any version below 3% usage has been removed to keep things readable.
|PHP 5.3.10||490350||11.92%||PHP 5.4||2069021||50.31%|
|PHP 5.3.3||419871||10.21%||PHP 5.3||1533073||37.28%|
|PHP 5.4.20||387450||9.42%||PHP 5.5||510596||12.41%|
|PHP 5.5.9||2475970||21.42%||PHP 5.5||5647892||48.87%|
|PHP 5.4.4||1022498||8.85%||PHP 5.4||3305929||28.61%|
|PHP 5.5.17||678997||5.88%||PHP 5.3||1716653||14.85%|
|PHP 5.5.16||529227||4.58%||PHP 5.6||886260||7.67%|
A few observations: 5.3 really is on the way out, 5.5 is now the major platform, 5.6 adoption is lagging behind last year's 5.5 adoption a little bit which is unfortunate but can perhaps be explained by the fact Ubuntu 14.04 ships with 5.5.9
PHP requirements in Packages
A second dataset I looked at is which versions are required by all the PHP packages present on packagist. I split it in two groups: those that required a given version at any point and those that require it in their current master version.
PHP Requirements - Anytime - November 2014
PHP Requirements - Current Master - November 2014
A few observations: almost 7% of packages that required 5.3 and 15% of those requiring 5.2 gave these up for higher requirements, that's pretty low but it's a start. 5.4 is getting a good foothold but 5.5/5.6 features are hardly used by OSS packages.
Again these numbers need to be taken with a grain of salt, but looking at the ecosystem from this perspective I would say PHP 5.3 can be safely dropped by most libraries and 5.5 sounds like a promising target!
TL;DR: New shiny product to support Composer development: Toran Proxy
A bit of history
I have spent quite a large part of the last three years working on both Composer and Packagist. This has been great fun for the most part, I learned tons, met a gazillion people both online and offline, got to travel places to talk about it at conferences. One question I get asked frequently is how I find the time to work on this for free, and my answer until recently was along the lines of: "I run my own business, which affords me quite some flexibility".
The problem is that I can not use this answer anymore. We have changed Nelmio's business model a few months ago to focus more on consulting and contracting. While this does not change the amount of time I can decide to spend on open source, it means that if I keep working the way I did in the past I will have to move under a bridge sooner or later.
Open-Source or a Salary?
I have spent quite some time over the last few months evaluating options to get out of the current situation. Having to choose between working on open source or earning a living is not great, and I feel horrible for ignoring GitHub issues and such. So the one thing I settled on is to work on an product around Composer that helps businesses using it but does not take anything away from the regular users.
Introducing Toran Proxy
Toran Proxy does mainly two things at the moment: it proxies Packagist, GitHub and others to make Composer faster and more reliable, and it allows you to host private packages easily. This was already sort of possible with Satis and many people used various amounts of hackery to get there. Toran makes it easier and faster. It integrates better with Composer since it acts as a real proxy. It can fetch/build packages lazily or can build them ahead of time, enabling you to choose between a lower disk usage or more safety against GitHub outages and such.
It comes with a yearly license fee, which includes updates and will hopefully allow me to work full time on improving Composer & Packagist. There is quite a big backlog of issues to look at and pull requests to review, finalize and merge. Packagist also has tons of potential for improvement. I have a million ideas and I really hope I get the chance to work on them. For example improvements in the discoverability of packages alone would benefit most everyone in the PHP community.
Of course Composer and Packagist remain free to use and fully open source. There is no way that will change. I just hope I can continue to work on them, for the community and supported by the community. Check it out!
Up until today if you run a home-grown package repository serving private packages it was quite a pain to use with Composer. You did not have efficient way to password-protect the repository except by inlining the password in the composer.json or by typing the username/password every single time.
With the merge of PR#1862 and some further improvements you can now remove credentials from your composer.json! The first time Composer needs to authenticate against some domain it will prompt you for a username/password and then you will be asked whether you want to store it. The storage can be done either globally in the
COMPOSER_HOME/auth.json file (COMPOSER_HOME defaults to
%APPDATA%/Composer on Windows) or also in the project directory directly sitting besides your composer.json.
You can also configure these by hand using the config command if you need to configure a production machine to be able to run non-interactive installs. For example to enter credentials for example.org one could type:
composer config http-basic.example.org username password
That will store it in the current directory's auth.json, but if you want it available globally you can use the
The advantage of having it in a separate file is that you can easily add this auth.json to .gitignore and let every developer in your company have their own credentials in there.
And I did not forget the security-minded folks that do not want to store anything on disk and do not want to be prompted every time! You can use
composer config -g store-auths false
Altogether these small improvements should make some use cases much easier so that is great news.
Here is a short update on the conferences I will attend in the coming months, in case you want to meet or if you are just looking for a good one to attend:
Next month I am very happy to have the chance to visit Tunisia for the first time and go speak about Composer at the Symfony Tunisia event. It will also be the first time I give a talk in French, which freaks me out a bit since I am so used to speaking about technical matters in English. You can therefore expect a bastardized language known as Frenglish.
Right after that is the first SymfonyCon happening in Europe, which will be in Warsaw, Poland. And I hear it is almost sold out so in case you were thinking of going hurry up and grab a ticket now.
In January I will head back to my mother nation of Belgium (insert "while it is still a country" joke) for two events: first I will speak at PHPBenelux about web application monitoring with Heka. Then the weekend after there is FOSDEM where I hope I can also meet a few people from outside the PHP community.
Why? To keep it short, Liip is a great company to be employed at - and they're hiring - but both Pierre and I have had the urge to be our own bosses for a while, and that is something that's hard to suppress. Eventually we had to give in.
Slippy is a HTML Presentation library written with jQuery, it takes a html file in and plays it in any browser.
It is optimal for programming-related talks since it includes a syntax highlighter and is very easy to use since it's just standard html markup with a few classes to enable specific functions.
Obviously feedback is very much welcome and even though it's not perfect yet, I hope it'll be useful to others. More docs and styling fixes (the dark grey background wasn't too visible on a projector, my bad) should come soon as I have more talks planned, and the slide repository page will receive some love as well when I have time.
I would like to attract everyone's attention on the 0.3-alpha release of Arbit.
For those that do not know Arbit yet, it is a project management and issue tracker build in PHP. It uses CouchDB as a storage backend by default but work to support RDBMS via PDO is in progress.
Interestingly, it also provides experimental support for continuous integration, also fully PHP-based, unlike other popular solutions. This is not enabled by default in this release since it isn't fully ready but feel free to stop by the irc channel (#arbit on freenode) to know more.
The full announcement contains details about what we fixed and implemented in the 0.3.
As all open source projects, Arbit needs your help, I joined the project early this year and we have had a few contributions from other people since then, but we can always use more help. Therefore if you are interested and wish to take part by developing new features, fixing bugs or at least reporting them, please don't hesitate and get in touch! And as Elizabeth Naramore's article recently pointed out most people are afraid to contribute, I would like to say that no matter how skilled you are, contributions are welcomed. We will provide assistance if needed.
In recent news, this site got a new design, I thought I could make the content more readable and accessible, so I killed my old templates and style sheets and started from scratch, without photoshop this time.
There is also mobile browser (android/iphone) support which is by the way achieved with this very interesting CSS media instruction:
<link rel="stylesheet" type="text/css" href="/mobile.css" media="only screen and (max-device-width: 800px)" />
This means any device with a monitor less than or exactly 800px wide will load the mobile.css file on top of the default one. Note that using media="handheld" is not working for recent smartphones that consider themselves greater than old school internet-enabled cellphones, so this is the only way to do it.
Any feedback, especially bad, is appreciated.