Well that was a FUN experience upgrading one of my clients Magento stores from version 1.6 to 22.214.171.124. Magento is one of those platforms that seems to instil the fear of dread when upgrading core functionality and it working as intended.
Never have I had a smooth upgrade experience with Magento CE (community edition), not like WordPress, which for the most part is a breeze.
Part of the problem I have found, is that any upgrade issues never show their ugly heads at the point of being upgraded, and only later on they peek up above the surface.
In this particular case, the upgrade process appeared to be super smooth! No problems at all, and the console reported that everything had been upgraded successfully. I sat back thinking, wow that was easy.. what on earth was I worrying about.
My heart sunk, when a couple of days later my client contacted me asking about their order statuses not being updated and that they received this unusual email from PayPal. With the subject:
PayPal Instant Payment Notification Warning
After searching all the error logs in the var/log folder, the website root and the server logs, there was nothing to find! Nothing relating to PayPal IPN, any Curl communication errors, 503’s, 500’s, nothing!
A frustrated hunter I was, with a fully loaded gun and no trail to follow.
After contacting PayPal technical support team and WebHostingUK my hosting provider for additional support, I got the usual back-n-forth tennis match.
PayPal kept reporting the server was issuing a 503 (down for maintenance) and my host reassured me, that the server was responding properly to Curl requests and they could find nothing in the logs.
Unless I narrowed down the cause of this issue, I knew we were never going to make any headway and solve this problem.
Honestly, not much! I had hopped on to PayPal’s IPN Simulator, entered the clients IPN address and submitted some test data. Each time it returned an elusive HTTP error, which as you can see below does not help!
So I needed to know what was going on and went digging through Magento’s core files, specifically:
Everything appeared to look ok, when compared to the recently downloaded source files, I even added in additional lines of code at each stage of the logic to output the data to custom logs:
So we were receiving the POST data from PayPals IPN Simulator in the custom log. At this stage our Magento listener should send the unaltered data back to PayPal for verification, which should reply with a single word “VERIFIED” or “INVALID”.
I was sure at this stage that the listener was not sending back the data to PayPal. However, they continually reported that when they manually sent Curl commands to our IPN address, all they would receive were 503 and 500 error codes from our server.
Ok, so here we go.. I found several really interesting blog posts on the subject, with some interesting ideas and tried all of them!
None of the suggestions made on each of these blogs actually worked in this case, whilst I found them all very interesting and indeed helpful in my education on how the IPN listener works.
Ok, if you’re not bored already and have continued reading, I managed to solve the problem without support from PayPal or my web host. Whilst they were both being very helpful and asking for information from each other on things they could try, we were not making any progress.
I had got to the point of exhaustion, absolutely fed up trying to pinpoint the location of the error. It was gone 8pm and my lovely wife had made dinner, whilst doing her best to help me unwind. I decided to simply overwrite all the files via FTP in the folder: app/code/core/mage/paypal/ from the original source and come back and work on it later.
Only an hour later, whilst watching a film the IPN notifications were coming through… YAY!
So being honest, it was more of a blind spray and pray, a last ditched attempt to kill this bug… and SPLAT it went! I can only deduce that either there was a corrupted or missing file, as part of the incomplete process.
In hindsight, I wish I had tried this sooner, but you live and learn.
Not that I imagine my particular problem is very common, I went to PayPal and WebHostingUK with a debrief of sorts, on how the issue was resolved. I asked PayPal to pass on my ticket to eBay/Magento for future reference.
But I wanted to update them both to mysterious 503 error, that it turns out was part of the response generated in the code here:
There’s a line of code that sends a 503 response if there is an exception when receiving the data:
If they had known this, it might have reduced the amount of back-n-forth emails and helped get to the root of the problem sooner. Being that it was not the server reporting the 503 per-say, but the code itself.
Thanks for reading!