While some features have been/were impacted, tracking is not affected. All tracking on the Awin network is working correctly, no data was lost, and all sales have been recorded.
Our engineering teams are working hard to resume normal service across all features, and we expect everything to be resolved by this coming Monday. The teams are also working on fixing the root cause of the issue, as well as taking steps to prevent the infrastructure issues from reoccurring.
The infrastructure issues (the cause of which we’re still investigating) caused a delay in reporting by blocking systems from being able to process transactions. Following this, there was an additional infrastructure issue where data from our master database was not being replicated to other databases. This meant features that relied on getting data from these other databases were not able to access up-to-date data.
The following services were impacted:
Reporting
Over the weekend, transaction reporting on the Awin platform was delayed up to three days. All reports apart from the click ref and product performance report on the Awin platform are now up-to-date. While we are experiencing some technical difficulties in displaying data within the click ref report, no data has been lost. Publishers are still able to get click ref data from individual transactions from the transaction report export. We estimate the click ref report will be fully functioning again by the end of this week.
Additionally, there was an extended delay in reporting of cross-device transactions. This has now been resolved. Advertiser and publishers are able to view these transactions within the transactions report.
Transaction notifications
Since Monday, there was a delay on sending out transaction notifications. We have since processed this backlog and normal service has resumed.
Shortened links
Short links created on Monday and Tuesday were redirecting to a blank page. The creation of new shortened links both in the UI link builder and MyAwin chrome plugin have since been disabled. We've made further fixes to any links created within this period so they are now functioning correctly again. We will re-enable the short-link functionality once the master database issue has been resolved.
Payment Run
Today's payment run was successfully completed with no issues. We also expect tomorrow's scheduled invoice run to be completed with no issues. Reporting on program balances is currently affected and may display out of date data, we are working to correct this.
We apologize for any inconvenience this has caused to clients during this period, and that issues weren't communicated to clients in a timely manner. We have reiterated our crisis communication plan to ensure that in future any impact on services are communicated as quickly as possible. If you have any questions or feedback, feel free to get in touch with your account representative or contact us here.
Update 2/19:
We experienced another issue with transaction reporting in the evening on Sunday, February 18. This was caused by an automated pause in our transaction processing system. While our engineers continue to investigate the root cause of the trigger, they are working on restarting the processing system and reprocessing all transactions that have taken place between Sunday evening and now.
As a result, reporting will not be up to date until we have successfully reprocessed this data. As a side effect, you may also see a duplication of click data in your reports. This is temporary, our engineers are aware of this and any duplicate records will be removed once the process is complete. We aim to have our systems up to speed again as quickly as possible and would like to apologize for any inconvenience caused during this period.
Update 2/20:
We are currently experiencing delays with our transaction reporting across the interface, with incorrect data showing from around February 10 onwards. Click ref reporting and product performance reports are also showing no data from February 6 onwards.
Our engineers have worked through the night and continue to work on bringing our databases back up to speed as quickly as possible. We will continue to post further updates as to when this looks to be rectified, and we should have a clearer view on this by tomorrow (Wednesday, February 21.) We are also aware there is a duplication of click tracking that will be causing further incorrect data to show. However, our engineers are aware of this and will rectify the figures as soon as transactions are up to date and the databases are stable again. We can confirm tracking is not affected by these processing delays.
Update 2/21:
We have made considerable progress over the last 24 hours to bring our databases back up to speed. However, there will remain some discrepancies within our transaction reporting until all systems have been fully recovered and stabilized. This may cause some fluctuations to be seen in the data shown in reports during this period. We can confirm tracking is not affected by these processing delays and no data has been lost. Click ref reporting and product performance reports continue to show no data from February 6 onwards, and our engineers are working to fix this as soon as possible.
We anticipate that the majority of reports will return to normal by Friday afternoon (February 23.)
Update 2/22:
Our systems continue to bring themselves back up to speed, and our databases have made significant progress. Transactions are now fully caught up, however there will remain some discrepancies in reporting until all databases have returned to normal function. Our engineers are working on a fix for click ref reporting to be back up and running as soon as possible, alongside active deduplication of click data from February 1 - 20. We anticipate the majority of reporting will return to normal by Friday afternoon, and click ref reporting to return to normal by Monday.
Update 2/26:
Transactional reporting and click ref reporting are now up to date and running as normal. Product performance reporting is not currently showing data from February 21 onwards, however, this should return to normal later today. Tracking has been unaffected during this period. That said, our engineers are continuing to double check data today. Duplicate click data has been removed and conversion rates and volumes should also have returned to normal. We would like to apologize for the prolonged inconvenience this has caused and thank you for your patience during this period.
If you have any further queries or concerns, please get in touch with marketing@awin.com.
Awin CTO Peter Loveday provides an explanation to the platform issues. Read here.
FAQs
Q. What is the date range that is currently affected?
A. We are currently seeing reporting delays from February 10 onwards for transaction reports, click ref and product performance reports have no data showing from February 6 onwards.
Q. Which reports are behind and/or contain incorrect figures?
A. All reporting is currently delayed. Click reference and product performance reports that are not currently functional for data after February 6.
Q. What is the ETA on all issues and reports being fixed?
A. We are working to rectify the issues as quickly as possible and we anticipate to have an update on when the reports will be fixed by tomorrow (Wednesday, February 21.)
