blog.johanv.org (Posts about symfony)

Upgrading Wdebelek to krakboem 1.0

Johan Vervloet — Sat, 10 Feb 2024 14:42:00 GMT

As I mentioned in a previous blog post, I recently released version 1.0 of rva-vzw/krakboem, the custom php library I created to use event sourcing and CQRS in my applications. It comes together with a Symfony bundle, rva-vzw/krakboem-bundle, which is now at version 0.1.2.

The first project to use version 1.0, was dikdikdik, the score app for the wiezen card game. This is one of my pet projects, in fact the one that I regulary work on. So in order to test if the krakboem-bundle and the updated krakboem actually worked, I tried to use them for dikdikdik. As everything seemed to work fine, I tagged it krakboem 1.0.

After that, I wanted to use krakboem 1.0 in wdebelek, the app I wrote in covid-times, to play cards online.

Wdebelek also uses krakboem, but it is different in that it uses event based entity repositories instead of deciders. So I had to figure out if this would work out fine as well. And while doing he upgrade, I had a nice chance to improve the upgrade instructions for krakboem 1.0.

I can't say the update was an easy one. Krakboem 1 requires php 8.3 and Symfony serializer 6. And for krakboem-bundle, I need Symfony 6.4. All this means that I had to update some other dependencies as well. For php 8.3, it mainly came down to updating a package here and there. But since until last week, Wdebelek was built on Symfony 5.3, I had quite some work with the Symfony upgrades. (update 2024-02-14: Especially the new recipes for codeception 5 were not straightforward.)

The update of kraboem, from v 0.11 to 1.0, was also not trivial. Lots of classes moved around. And quite some abstract base classes became read-only, a concept I wasn't aware of back in 2020. (update 2024-02-15: Identifiers are now expected to be non-empty as well. This was also quiet an update.)

But since last week, the 4th of february to be exact, wdebelek is running with the latest versions of my libraries. It still needed a blog post, but hey, here it is, so I am a happy programmer again. (Writing the blog post took me a week, because of, you know, life.)

If you want to try playing cards with Wdebelek, you have to find at least 2 other people to play, and go to kaart.rijkvanafdronk.be. Remember that you need to drag the cards to play them. (This is not really clear from the UI, and it often causes some confusion).

I still have one project to update: 1jg, another score app for another game. This will probably a little easier than the Wdebelek update, because 1jg is already on Symfony 6.3. And of course I now have nice upgrade instructions for krakboem and its bundle now 😉.

Krak? Boem! My first Symfony Bundle

Johan Vervloet — Sat, 13 Jan 2024 20:52:00 GMT

Earilier this week, I published my first Symfony bundle. Should you use it? I don't think so. But I'm proud that I unlocked this achievement 🏆🔓

A little background.

Back in 2019, I started writing an application that keeps track of the points when playing whist. I called it dikdikdik, a typical bad name for an open source project. It refers to some catch phrase in Dutch, and it can be used at score.rijkvanafdronk.be. It has become rather popular over time, which I find very cool.

So ok, I created this app because I wanted something to help us with the scores of our card game. But maybe at least as important: I wanted to create an app to fiddle around, and to learn new things. I wanted to use Symfony and event sourcing, two technologies that were still rather new to me. After some time, I had a fun project, and I talked about it in a lightning talk at the last PhpBenelux conference (ever? 😢) in 2020.

Not long after that, in 2020, Covid broke out, and we couldn't play cards anymore because of the restrictions. So we needed an app to play cards online. Because I wanted that one to be event sourced as well, and because I didn't want to write everything again, I extracted all relevant classes and interfaces to a php package. rva-vzw/krakboem was born.

This package worked quite well for a couple of years. But things have changed. Back then, I used php 7.4. Now there's php 8.3, with final and readonly. I know more about the relationship between Traversable and Generator. I also have better understanding about what infrastructure code exactly is. And recently I learned how to use deciders, and krakboem didn't know deciders.

So krakboem could use an update, and that's what I did. I rearranged the classes, added final and readonly, introduced deciders. And while doing this, I also extracted all symfony messenger and doctrine code, and moved that to a bundle. So now I have a krakboem-bundle. And I also tagged version 1.0 of the base krakboem package 🎉

Not that I'm 100% happy with how krakboem is structured, but I didn't want to move every class around. And I'm fairly sure that it will be useful again for a couple of years without needing lots of changes.

The bundle is still experimental (version 0.1.2 at the moment); I don't really know yet how to properly create a bundle. But I am happy, because it already takes care of the dependency injection for the doctrine and messenger infrastructure.

So next thing on my to-do will be: upgrade my other krakboem-based projects, so that they all use my new bundle. I don't expect this to be a whole lot of work, but I don't have a whole lot of time either. So we'll see how it works out.

Whist with a decider

Johan Vervloet — Thu, 07 Sep 2023 18:47:00 GMT

Back in June, I attended DDD Europe 2023. It was the first time I was at this conference, but I liked it a lot, and I was intrigued by Jérémie Chassaing's talk, in which he used a decider to keep track of the internal state of aggregates.

I don't know a lot (yet?) about functional programming, but I was charmed by this elegant way to work with commands and events. So I wanted to experiment with deciders as well. As it turns out, I have an ideal project for doing this kind of experiments: dikdikdik, my web based score sheet for the wiezen (whist) card game. It already had commands and events, so using a decider should not be too hard.

Dikdikdik has two aggregates: a table (as in a piece of furniture, the table is where the players sit down, and play their games) and a score sheet. Initially those aggregates had methods that emitted events, and other methods that applied those events to their internal states. The latter ones were also used to rebuild the aggregates based on their event streams, since dikdikdik is also an event sourced application.

In a first step, I replaced the table entity by a TableDecider and a TableInternalState class. The idea is that when you pass the internal state of a table and a command to the decide-method of the TableDecider, it produces events, describing what happens. When passing those events with the internal state to the evolve-method of the decider, it applies the events, creating an updated internal state.

I could use this decider pattern to implement event sourcing, and it also allowed me to create a kind of testing framework, that made it easy to create given-when-then-unit tests in a quite elegant way, see e.g. this test that tests joining players.

I created the decider in a more-or-less test driven way, by converting my old tests for the Table aggregate to new tests for the table decider, and then make them pass by converting the old logic to the decider based logic.

This conversion was interesting, because it made me look back into the existing code for the write side of the table. And whenever I see code I wrote a couple of months or years ago, I am reminded about the things I learned since then. That's a good thing, I presume. Other pieces of the code have become less relevant, since the project has changed as well during the last couples of years. I created a couple of issues on gitlab for the oddities I enountered, e.g. #304, #308, #302.

When all unit tests passed, it didn't take a lot of work to make the integration tests and e2e tests to be all green as well. Which made me happy, because I think this was an indication that the degree of decoupling in my project is low. If you look at the merge request (which I reviewed en merged myself, since I am the only developer on the project 😉), you will notice that almost all changes are in the Domain\WriteModel\Table namespace, and almost no other things needed to change. The other aggregate, the ScoreSheet, still uses my old way of working, and guess what: that doesn't matter. The score sheet doesn't need to care about the internal workings of the table, and vice versa, the events are the only things that pass the boundries between them.

All with all, there were two other things I needed to change. One thing, was the validation of the games that can be logged. Now I can use a service for this, that I can nicely inject into the decider, which is way more elegant than what I did before to let the aggregate class validate the logged games. (The validation service still has some issues in its current form, but it is already injected; that's something.)

Another thing is that the decider is aware of the initial state of the aggregate. Previously, I prepared the initial state by handling the event TableCleared. Now this event is not really needed anymore, but in my integration tests I still (ab)use it to reset the read models.

Another interesting change that happened, when introducing the decider, is that I don't have a dedicated handler for each command anymore. Which feels a little strange to me, I always learnt that each command should have its own handler. Now the TableDecider takes every TableCommand, and calls the handler function that corresponds to the command. But I guess these functions are the handlers now, which may be fine.

Anyway, I am happy about the result I've got so far. I think I will already release this to score.rijkvanafdronk.be at the end of this month, so that we can try it out during the next meetup of our wiezen club. (Of course everything should just work, since all tests are green, but I want to use the application quickly after the release, so that we would notice any bugs that are not covered by the tests.)

So what's next:

Now that the write side for the table is handled by a decider, I will also replace the one for the score sheet. Since the ScoreSheet class is much smaller than the Table class was, this should take less time than converting the table, but I also have little free time to develop these days, so we will see how this turns out.

And then there's the process manager, that passes commands to the score sheet when games are played at the table. I think Jérémy did something like this in his talk, by combining deciders, but I will probably have to watch a recording of his talk again, because I'm not sure anymore about how this works.

Another thing I want to do, is add the generic decider classes I created to the krakboem libray I created for my own event sourced projecs. That library needs some updates as well, because it was created when php 7.4 was a thing.

For a short period of time, I had timelines for the new features of dikdikdik. But not anymore, because life happens, and I will see when I have time to code. Anyway, I want to thank Jérémie for his inspiring talk. I'm glad I got this far already, and we'll see what the future brings.

Until then: have a nice time playing whist.

Migrating a web app from VueJS to Symfony UX: retrospective

Johan Vervloet — Sun, 07 Nov 2021 20:34:00 GMT

The last couple of weeks, I have been refactoring the frontend of dikdikdik, one of my pet projects, that we use to track the scores when we play the card game 'wiezen'.

The release of version 3.0, in which I threw out VueJS in favour of Symfony Turbo and Symfony Live Components was announced yesterday, and in this blog post I look back on the refactoring, making some kind of roundup on what I like and dislike about the new frontend I created with those new symfony-ux tools.

Combining Symfony live components and Mercure

Johan Vervloet — Wed, 27 Oct 2021 19:10:00 GMT

Here's the next blog post of my little series about Symfony live components. Last week, I wrote my first Symfony live component, and today I will use Symfony Turbo Streams (with mercure) to update my components.

My First Symfony Live Component

Johan Vervloet — Wed, 20 Oct 2021 19:23:00 GMT

As I mentioned yesterday, I want to use Symfony Twig Components (and even better: Symfony Live Components) to rebuild the frontend of our wiezen score app, dikdikdik.

Today I am very happy, because I created my first live component 🎉🎉🎉. It works, and it is very cool, because now I can use PHP to describe the behavior of a front-end component.

Trying out Symfony Twig Components

Johan Vervloet — Mon, 18 Oct 2021 20:18:00 GMT

Last summer, I attended SymfonyWorld Online 2021 Summer Edition, and I was fascinated by Ryan Weavers talk about Symfony Twig Components.

As the README says on the github project page

Twig components give you the power to bind an object to a template, making it easier to render and re-use small template "units" - like an "alert", markup for a modal, or a category sidebar.

I found the presentation very promising, and I wanted to try using twig components in combination with stimulus for the frontend of dikdikdik, a hobby webapp I maintain, to keep track of the score for the wiezen card game.

Updating your Symfony app from Mercure 0.10 to Mercure 0.11

Johan Vervloet — Sat, 26 Jun 2021 05:59:00 GMT

Last wednesday I updated dikdikdik, a score sheet app for solo whist, so that it now uses Mercure v0.11, instead of v0.10. I use docker-compose for the development environment of dikdikdik, and GitLab CI/CD to run the end-to-end tests with codeception and selenium.

This was not a trivial update, because v0.11 was a major milestone for the Mercure project. In retrospect, I didn't have to change much, but it took me some time to find out what exactly I had to change, especially to make the end-to-end tests running again.

Selenium web tests on gitlab-ci for an app using Mercure

Johan Vervloet — Fri, 08 May 2020 20:15:00 GMT

I recently learned how to use Mercure so that the php/Symfony back-end of my application can send out events that can be picked up by the javascript on my front end. This is really cool technology, you should definitely try this out if you haven't already done so.

Now if you write this kind of software, of course you also want to write some web tests, so that you can automatically check if existing features of your program keep working after you add new things.

For these kind of tests, I use gitlab-ci and codeception, as I wrote in previous posts. Today I got these test working with a Mercure hub.

You need to use a mercure service in the job running your web tests, which you can configure in .gitlab-ci.yml:

Update 2020-12-03: I use the 0.10 version of the mercure container, because the acceptance test job started failing with 0.11, with these error messages:

Message: "Couldn't connect to server for "http://mercure:3000/.well-known/mercure"." In HandleMessageMiddleware.php line 80:

Couldn't connect to server for "http://mercure:3000/.well-known/mercure".)

So here's the adapted .gitlab-ci.yml-file:

variables:
  JWT_KEY: YourSecretKey
  DEMO: 1
  ALLOW_ANONYMOUS: 1
  HEARTBEAT_INTERVAL: 30s
  ADDR: ':3000'
  CORS_ALLOWED_ORIGINS: '*'
  PUBLISH_ALLOWED_ORIGINS: '*'

# ...

acceptance tests:
  # I would prefer to use a public php-7.3 image with mysqli, but
  # I don't think such a thing exists.
  # So let's reuse the image I created for the dikdikdik web tests.
  image: registry.gitlab.com/rva-vzw/dikdikdik.php
  services:
    - name: mariadb:10.4
      alias: db
    - name: selenium/standalone-chrome
      alias: chrome
    - name: dunglas/mercure
      alias: mercure:v0.10
  # (the rest of the job follows here)

The JWT_KEY should work with the JWT your backend uses to publish events. The port number in ADDR as well. If you use Symfony, as I do, the corresponding configuration of your backend should be in the .env file, e.g.

MERCURE_PUBLISH_URL=http://mercure:3000/.well-known/mercure
MERCURE_JWT_TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJtZXJjdXJlIjp7InB1Ymxpc2giOltdfX0.FFSjymJCGRDWrmAmPJDoVGuYwnx5FRTjRFkkYfvLkUg

(Needless to say: don't use the production JWT and token in the source code that is used by gitlab! If you're unsure how to create a JWT, see the section ‘Creating your JWT’ of this excellent blog post.)

Update 2020-12-03: That blog post doesn't exist anymore, but I found another page that covers mercure and the JWT token.

Now the problem I ran into, is similar to the general problem I had with web tests: the chrome container doesn't know where the mercure hub is; the host name ‘mercure’ is not resolved to an ip address.

Update 2022-06-20: The solution described below will still work, but I found a better one. You should just set the FF_NETWORK_PER_BUILD variable to 1 in your .gitlab-ci.yml file, as described in this post. If you want to see how this works for a real life project, you can check out the .gitlab-ci.yml file of dikdikdik, an app to keep track of the scores when playing solo whist.

To fix this, I figure out the IP address of the Mercure hub when I start the job, and I replace the host name of the MERCURE_PUBLISH_URL in the .env file by this IP-address. This works, since my frontend retrieves the publish url from the backend by a rest-call.

This is how that looks like:

  script:
    - MERCURE_IP=$(getent hosts mercure | awk '{ print $1 }')
    - echo Mercure IP $MERCURE_IP
    - sed -i "s/mercure:3000/$MERCURE_IP:3000/" .env

You can review the complete job definition in the .gitlab-ci.yml file in the wdebelek repository. The file acceptance/TrickCest.php contains an example test involving server sent events. And hopefully you can see in the pipelines that all tests passed without problems 🤓.

Dependency hell when upgrading symfony from 4.3 to 4.4

Johan Vervloet — Fri, 10 Apr 2020 19:19:00 GMT

Recently I tried to upgrade the symfony version of the dikdikdik wiezen score app from 4.3 to 4.4. As usual, I ended up in dependency hell.

Here's what I got:

$ composer update symfony/*
Loading composer repositories with package information
Updating dependencies (including require-dev)
Restricting packages listed in "symfony/symfony" to "4.4.*"
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Conclusion: don't install symfony/validator v4.4.7
    - Conclusion: don't install symfony/validator v4.4.6
    - Conclusion: don't install symfony/validator v4.4.5
    - Conclusion: don't install symfony/validator v4.4.4
    - Conclusion: don't install symfony/validator v4.4.3
    - Conclusion: don't install symfony/validator v4.4.2
    - Conclusion: don't install symfony/validator v4.4.1
    - symfony/validator v4.4.0 conflicts with doctrine/lexer[v1.0.1].
    - symfony/validator v4.4.0 conflicts with doctrine/lexer[v1.0.1].
    - symfony/validator v4.4.0 conflicts with doctrine/lexer[v1.0.1].
    - Installation request for symfony/validator 4.4.* -> satisfiable by symfony/validator[v4.4.0, v4.4.1, v4.4.2, v4.4.3, v4.4.4, v4.4.5, v4.4.6, v4.4.7].
    - Installation request for doctrine/lexer (locked at v1.0.1) -> satisfiable by doctrine/lexer[v1.0.1].

I ran composer require doctrine/lexer, hoping that this would give me some clues:

$ composer depends doctrine/lexer
doctrine/annotations  v1.8.0   requires  doctrine/lexer (1.*)
doctrine/common       v2.10.0  requires  doctrine/lexer (^1.0)

I guess that doctrine must have been the problem.

So I uninstalled doctrine (composer remove symfony/orm-pack), upgraded symfony (that worked, 🎉), put the doctrine configuration files back in place, and reinstalled doctrine (composer require symfony/orm-pack).

It did update something, but then I got this error:

Executing script cache:clear [KO]
 [KO]
Script cache:clear returned with error code 255
!!  PHP Fatal error:  Uncaught Symfony\Component\Debug\Exception\ClassNotFoundException: Attempted to load class "DoctrineCacheBundle" from namespace "Doctrine\Bundle\DoctrineCacheBundle".
!!  Did you forget a "use" statement for another namespace?

After some searching on the internet, I found out that I had to remove the line

Doctrine\Bundle\DoctrineCacheBundle\DoctrineCacheBundle::class => ['all' => true],

from app.php to fix that one.

Of course now I had various other issues, but the dependency hell was fixed, so I am glad I can share this with you.