Still alive!
Wow… more than one year since my last post. Time flies, especially if your company got aquired and you experience suddenly very exciting adventures. What a year!
Anyway, back to business - I’ll increase slowly my public appearance and therefore I am happy to announce these two upcoming events:
1. I am attending next week a MASS Technology Leadership Council Breakfast Seminar in Cambridge, MA where I’ll speak about “the Secrets to Achieving Unprecedented Rates of Growth and Innovation” at KAYAK 
2. I got another chance to visit beautiful Hamburg - always worth a visit where I’ll run two Sessions (in German) at the Devcon Hamburg.
Once again - Agile Development with PHP and an interesting session about our Caching/Sharding/Distributing Strategies @ swoodoo.com when getting hammered by TV traffic.
Your choice - Boston or Hamburg
Upcoming talks…
After my discussion with Arno about the right way to develop software I’ve decided that it would be a good chance to meet him in person while visiting PHP Con in Barcelona. And besides that, Barcelona and the PHP Folks there is always good for a visit.
Therefore there is - once again - another chance to enjoy my talks called ‘Refactor it! A practical journey into the test-driven world’ and ‘Agile Development with PHP in Practice’ . I did run both topics already quite often but obviously there is still a need for it. If you are new to agile development or interested in refactoring your old (shitty?) sourcecode - come in and join.
And if you are interested in the topic of the discussion with Arno - feel free to contact me. I will organize a meeting with where we will discuss the whole item over a few bottles of red wine. After receiving so many comments it would be nice if we get a few more opinions together.
After Barcelona Summer is definitly gone and it’s - as every year - time for the international PHP Conference in Karlsruhe. This time Thorsten and me will run a half-day Workshop called “Unittest for Dummies” which will address unit testing beginners. If you are not already into TDD you should definitly consider to come. Furthermore there will be a session about how swoodoo managed to withstand the massive amount of requests due to (*yeah*…) the big success. I’ve titled it “caching, sharding, distributing - Scaling best practices” - so you might get an idea about the contents.
That’s it - live from the “Oktoberfest” in munich
Extension Terror?
When I first saw the posting about ‘bad’ code in OXID over at phpterror I wondered if I could ignore this - but now I’ve realized that this article was published on Planet-PHP and even more - other people start to copy the content of the article.
As I am the guy who introduced the disliked functionality many years ago ( actually years before ZF popped up) I feel the need for a statement to put the things into the right order. Please note that I did work for OXID in the past (years ago) but I do not nowadays.
Arno criticised the way modules == classes are instantiated in OXID eShop. Actually I do believe that exactly this feature is the most coolest in OXID and should be implemented in more OSS.
Let’s start with the “why the fuck did the guy implement this”?
If you do run a website out there you know how important it is to keep it up-to-date with the latest patches due to security reasons. Now imagine - we are in eCommerce area. You deal with payments, credit card data and sensitive information about what people ordered from your shop. Not only Creditcard data is sensitive - imagine it would leak that you ordered the extra-big-boobs doll? You see - especially in this area you need to make absolutly sure that your servers and the software is safe.
Unfortunately you can’t start an eCommerce business out-of-the-box. You need to adopt the software to your needs and processes. Think of payment, ERP, delivery notifications, link to your stock, uploads to price-comparison sites etc. Some of the functionality you might need to develop yourself - for other stuff you might find already existing modules out there.
Therefore you need to change code, and/or install external modules which modify the functionality. In former days you simply edited the source code (as we are talking about OSS) and made manually sure that the modules you install are compatible. Don’t forget - each of them will work with the out-of-the-box shop - but they still should work after you added your changes and - also need to work after you installed some other external modules which might overwrite/change the same classes/functionality which your new module also want to change.
You end up with a highly customized shop, many changes, a lot of work to dig through the sourcecode of the installed modules to make sure that they don’t harm themself and… you lost the possibility to automatically appply patches/releases. Upon each patch or new release you need to manually redo/check your changes.
This sucks. It sucks a lot. And this is exactly why I’ve introduced the criticised module functionality.
So what did I do?
As Arno already copy&pasted the source I won’t repeat it here. But let me explain the main idea behind the concept.
As OXID is fully OOP you can change all the functionality by inheriting your class from any base class you might want to change. You add your changes and - et voila - you still can overwrite the bases classes with new releases/patches and your changes will work. This is called object oriented programming.
Let’s assume you want to change the method “getPrice” in the class “oxarticle”. You simply overwrite oxArticle::getPrice and… it would work if the system would know that your class exists. Therefore you need to register your class and let the OXID Framework know about it so that it automatically will instantiate your class instead of oxArticle each time the object is needed.
So far, so good, so what?
What happens if you install some other module ( e.g. from here) which will change the same method in same class? It would screw your changes and you would end up with manual changes again. To avoid this, the OXID Framework supports “chaining” of inheritance… and the order is set in the config. So you can define that the modules oxArticle implementation is executed before your class or after.
I do believe that by overriding the classes you do get the most flexible option to change everything without loosing the functionality to update the shop whenever you want. For sure this could be solved differently. Events or Hooks would have been a way to go - but this would involve a lot of additional coding == lines of code and therefore introduce new vectors for bugs. And - it is by far not as flexible as overriding the class. Nowadays it could be solved also with Reflection. But this is a bit too “magic” in my opinion. Therefore I still believe that the way I’ve chosen is the best way to solve the problem and - most likely - I would do it again today.
Last but not least - I would like to comment the show me your 94% Unit Test coverage! posting. I’ve contacted OXID already one year ago about this and I fully agree with Arno here. It sucks to advertise with some test-driven-development features and then keep the tests for yourself. OXID has to change this. Now.
Today I had a good skype discussion about the issue with Arno - I really like his style even though we both do not agree - and the result for me was that I’ve decided to submit a few proposals to the PHP Conference in Barcelona to get the chance to meet Arno and discuss the whole issue over a bottle of spanish Rioja. Or two. Or…
Where are my mock objects you lazy sons of b….
Bye Bye bella Roma
The last few days I had the pleasure to visit the PHP Conference in Rome. It was - as usual - really good organized and even though it was not that big as other conferences I personally think that it was quite successful. And for sure the location… thousands of years of culture. There is simply no other city in the world with the age and the cultural background like Rome.
As promised here are my presentations:
( V2 as this is an improved version now with better examples using OXID eShop to show the power of selfmade Nagios/Cacti plugins)
PHP@INDIA ?
Did you ever wonder if these Millions of Indian developer we often read about are capabale of doing good in PHP? I did. And I tried.
Back in my past (*cough* - hitting 40 soon…) I’ve outsourced parts of my jobs to many countries, starting with a company in Prague/Czech Republic together with my friend Matthew, and moving later on to lithuania where I still run (together with my mate wolfgang) a development office with a total of 60 developers, coding for various projects like swoodoo or oxid.
During these nearly 25 years I did work for sure in a few projects where coding was outsourced to india. My experience in the past was not really good. The classical project in india was:
- huge
- really huge
- f.. huge
did I mention huge?
Why is that so? Well - the average indian developer is (was!) teached to work as some sort of coding-ant. Don’t think, just do what you gotta do (futurama). Thus this ended in a culture where you need to write a very detailed spec ( thousands of pages) and hand this over to your “ant” farm where 500-1000 developers code what you told them todo. Still you need to check personally the quality as (in the past!) the indian culture does not support being honest in case of problems as this is considered rude. Ah - being rude… tell me.. the ones out there who had the pleasure… to meet me in person know that I am a really honest person thus it is very easy that myself is been taken as rude….
So this was the past - 12 years ago. In 2008 I’ve started a second try - and guess what ? It worked! After investigating the right partner via my personal network I’ve started to work with Anantara. A small company, with very bright minds in management, reliable and trustworthy. Nothing what you can expect anywhere on the planet.
Anantara was working in the usual areas - Java and .NET but not very much in PHP. So I helped them a bit to kickstart a fresh PHP team - and they kept their promises and delivered a good working team. To my pleasure they realized that the PHP area is a very interesting market to them, so they decided to put a lot of energy in this area - sic! Times are really changing, especially now when we talk about the crisis - it matters what you pay for your software, and PHP is simply a lot of cheaper (aka faster) developed compared to .NET and java.
Don’t get me wrong - outsourcing is always difficult and by far less efficient compared to in-house developers. You always have to bridge a cultural/business gap which is most likely there. You can’t expect that some people anywhere on the planet earth understand your business needs without teaching them thoroughly.
But - in the past I had much worse experiences - and now I’ve really enjoyed to work with Anantara the last year. Our project stopped now, as it ended succesfully, but I am definitly looking forward to work with these guys in future again. From a PHP skill level I would estimate that there is average know how - which is better than many code I’ve seen in the last years developed in germany. Needless to say that they do agile development - what else…;)
So - if you are interested in contacting them or you do have any other question concerning outsourcing - feel free to contact me. I’ve decided to support Anantara a bit as I really do like them - and no - I am not paid, nor am I a shareholder there. Most likely they will pay me a beer (or two…) 
IPC08 - This was 2008, looking forward to 2009
Again a year passed by and the International PHP Conference closed his doors. I finally made it home, slept 14 hours and now I feel not like a zombie any more. This year was quite exhausting for me as I had to stay to full length - from Sunday Evening till Friday Evening. And - as usual - the evenings were the most interesting part. Everyday (a bit) drunken… I think I need to train more to make a better position next year
As always “la famiglia” showed up so I met all the usual suspects starting with the guys (and girls) from gayflowr Mayflower to Tobias Schlitt and Kore which whom I had as usual a lot of fun till the very well known Derick Rethans and Sebastian Bergmann. Ah, not to forget Max, Markus and Gaylord Over a whisky Markus and myself discovered that the world is really small. We’ve never met, but one guy working for me on the very old times brought him into touch with PHP.
Interesting for me was also that I got into touch with a few really nice guys who I somehow missed to meet in the past. Pierre is a good example. I don’t know how I could miss to meet him I think we fit together as we might both start the PHP Senior Group - or maybe we build it alone ? Furthermore I’ve met Ralf and Nils who attenden PHP Conference even tough he seem to hate it Having beer with them together on the “late oktoberfest” was a pleasure.
So - enough about the party ( even tough I consider this the main part on any conference as there you get the real good answers and contacts ) - which talks I held this time ?
Together with Thorsten ( the ‘womanizer’ with the most famous blue eyes) we had a full day workshop on “Theory and practice – migrating your legacy code into our modern test driven development world”. For me very interesting was the even tough we expected that everybody who attended already uses TDD - this was not true so we had to change the workshop a bit on the fly and went back to the basics.
On Tuesday I continued giving a talk about Monitoring your business. I was not quite sure what the attendees expected but it seems that I could manage to satisfy them.
Finally I’ve agreed to talk on the last “Enterprise” Day about “Agile Softwareentwicklung bei Geschäftsanwendungen” - sorry about the german, but the whole day was in german. And many thanks to Periklis who gave me the very interesting figures in it about the performance gain a team can get using agile methods.
On Friday I also had to hop in for Bjoern who unfortunately got sick 
- hope you will be better soon Bjoern! It is quite challenging to create a talk in 40 minutes out of the nothing but with the help of Thorsten I made it. And so I’ve talked a bit about “So gelingt der Umstieg von PHP4 auf PHP5: Erneuerung von Geschäftsanwendungen unter Qualitätsaspekten”
I think this was it - at least everything what I can remember - if I met you and forgot to mention you, please kick ping me.
Refactoring your legacy code - Part four: Fixture this!
Another day in paradise. The right time to continue with our small series about refactoring. Today Tomas and me will talk a bit about the right (and wrong) way of preparing your data for the tests - the so called “Fixtures”. If you will read in the text the context “we” then Tomas wrote this part about his experience with the OXID refactoring.
Remember - we learned how to test simple 2×2 functions but when we tried to test more complex ones initialy we felt like stalled with no or negative progress. We tried to identify what is the difference between simple functions returning a value from supplied parameters and the real life ones.
The problem with those functions is that they depend in most cases on your application state (script globals, config vars, session vars, object state, existing data etc..).
As already shown in the previous postings we are able to separate all the code into a few abstraction layers (db access, fetching config vars,..) and we did that, but finally we still wanted to test in a simple way the high-level functions. Eg. createOrderFromBasket()->saveOrder()->loadOrder() flow. Lets identify the issues preventing us from simply testing these functions and see the way how OXID handled them.
Time consuming fixture creation:
It is quite easy and fast to prepare a small data set for tests, but when you try to test a bit more complex functions like the calculation of values in a shopping basket, then you have to prepare the full basket and define all the needed values (products, amounts, vat… ). And if you have to do this for every test, this is a pretty boring task.
OXID decided to solve this issue by creating a test database filled with all the common data: users, orders, various combination’s of products, discounts etc. For each test where we needed this data we created on the fly fixtures out of this database - some sort of “fixture” generator with a static source (our database). This helped a lot in speeding up fixture creation. I’ve heard some opinions and complaints that our big demo data is a big NO, not a really valid “fixture” and you should prepare fixtures for every test separately, but why not to consider our test data as one fixture - only bigger? Leave us a comment if you think different!
Actually the way described above violate the theory behind TDD. In the swoodoo team we do it a bit different - but still I want to share this with you, as it’s a real world example and - not to forget - a very successful one.
Test data is altered via another test:
And here we have the next problem. When you have one big test data chunk, it gets altered by other tests. One of the (very wrong) ways would be adjusting tests, depending on which order they are run. But it’s clear you will reach a dead end very soon. Do not even try this. Wrong!
How did OXID solve this? First of all we created a rule: use test data for reading relaxed (actually the nature of our business logic is that it is “read data” oriented) and - if we need to change test data for test – make sure we change it back on tearDown(). This is still dangerous as one failed test may result of other test failing. Tracking this kind of failures is most difficult task. So additionally we use a small script which is run after each test (test class), which diffs existing db with initial and restores it.
Actually the much better way is to create the fixtures each time manually with tools like yaml. This way is the correct way and it can be solved also without too much work if you create yourself a generator for yaml. Doing so will avoid many problems like the ones described above. On the other hand - this is pure theory. And refactoring an existing application is something totally different than creating a new one with the latest state of the art technology.
Refactoring your legacy code - Part Three: View me!
And there we are again - today we will take a deeper look into the views.
It is important to decide which classes are critical to test and which classes only deal with retrieving information. The MVC pattern is the answer. It separates logic from presentation. View classes only “reroute” the information and Controller classes host all the logic.
Decoupling Models from the Views is vital for your tests. For the model you need to have nice unit tests. The Views should be tested by acceptance tests. Some people do also run unit tests on the Views - it is possible - but my personal opinion on this issue is that you really should try to avoid logic in the view and then it’s sufficient to test them via selenium tests.
This is why we created a new rule in effect – no logics in view classes, just getters and setters for controllers and super global wrappers.
Let me give you a good hint: do never ever use the accessor methods for the super globals from the model classes. Let it be $REQUEST, $COOKIES or even $SESSION. All these parameter should be collected in the view and passed on to the model. Doing so gives you the chance to have small, clear and decoupled functions and also your tests can be done very easy.
Let’s see how this could look like before:
class displayUserDetails() { /** * Processes input and sends user first name, last name to display; */ function show() { global $dbLink; global $templateEngine; $itemId = (int) $_REQUEST['user_id']; $firstName = $dbLink->getOne("select first_name from users where id = $itemId"); $lastName = $dbLink->getOne("select last_name from users where id = $itemId"); $templateEngine->addTemplateVar('firstName', $firstName); $templateEngine->addTemplateVar('lastName', $lastName); $templateEngine->display(); } }
and after refactoring you should get something similar:
/** * A view class responsible for displaying user details. */ class userView() { /** * Loads user object and sends first name, last name to display */ public function show() { $userId = $this->_inputProcessor->getParameter("user_id"); $this->templateEngine->addTemplateVar('user', $this->model->loadUser(userId)); $this->templateEngine->display(); } } /** * And the corresponding model */ class userModel() { public function loadUser($userId) { $user = new User( $userId ); return array( 'firstName' => $user->getFirstName(), 'lastName' => $user->getLastName()); } }
And - as usual thanks to Tomas who provided this time the examples.
Refactoring your legacy code - Part Two: Global Problems ?
Today we want to address a global problem. No - not the global warming up - instead the misuse of global scope in variables and functions. This is a followup to my previous posting. And again (as every posting in this series) some advertisement for our talk at the phpconference.
Legacy Applications are often done in PHP4. Which means no really object orientation. All class variables are public - and this is how they are used. Over the time many members are set/get in different places.
If you start writing a test - you need to make sure that you only test specific functions. All subsequent calls, data etc. need to be stubbed == faked.
Quite common would be that you have to deal with many global variables. In OOP applications it shouldn’t be that the output of the function is influenced by global parameters. And for correct testing you need to refactor this.
Both cases need to be addressed by you - and this is where the refactoring starts. As I wrote in the last posting - while writing the test you have to refactor otherwise you can’t create proper tests.
But what to do with PHP super globals $_GET, $_POST, $_SERVER, $_COOKIE etc.? They are globals by PHP nature. You could create a wrapper (proxy) myInput::getParameter(‘getParam’) and use it everywhere in the code. No exceptions allowed. Doing so you can stub these calls also. And furthermore it adds a bit more security as you can add in this function the very basic and idiotic security checks. They won’t be enough for sure - but it’s a start.
The same applies to the global configuration parameter. The most important here is – you must be able to supply your own parameter in tests here. This way you can write tests for different request parameters and config options. On test tearDown() all parameters should be restored to defaults.
Let’s see how this could look like. You might find such settings in your code:
class someOtherClass { var $setting; function calculateSomething($a, $b) { return $a+$b; } } class myOldNastyClass { function needToTestThisFunction() { $class = new someOtherClass(); $z = $_GET['input']; // .... return $class->calculateSomething( $class->setting, $z); } }
This code has more or less all the pitfalls you will explore while dealing with globals. Before you can even think of writing a test for this you need to refactor it. And this could lead to:
class someOtherClass { private $setting; public function calculateSomething($a, $b) { return $a+$b; } public function setSetting($set) { $this->setting = $set; } public function getSetting() { return $this->setting; } } class myInput { public function getParameter($name) { return $_GET[$name]; } } class myOldNastyClass { private $input; // set e.g. in constructor public function needToTestThisFunction(someOtherClass &$class, $z) { $z = $input->getParameter('input'); // .... return $class->calculateSomething( $class->getSetting(), $z); } }
Now we have some code we can test. Let’s see how the tests could be:
class myStub extends someOtherClass { public function getSetting() { // fixture return 99; } } class myInputStub extends myInput { public function getParameter($name) { // fixture return 999; } } // in the test $test = myOldNastyClass->needToTestThisFunction( new myStub(), 88); // ....
I hope I could point you in the correct direction