How Flomatika helped DevMagic measure and improve team performance
Download the full reportDownload PDF
DevMagic, based in Brazil, is a development company that specialises in software solutions for local and international clients. DevMagic helps companies around the world rapidly develop digital products and MVPs for testing and validation in new markets, while also making the software development management process simple and transparent.
With eight years of operation and more than 90 projects under their belt, the team has one simple objective: to accelerate the growth of their clients’ businesses.
Favouring kanban and flow-based approaches, the DevMagic team are confident early adopters, quick to jump on new tools and techniques that enable fast, high quality development, and set them apart from their competitors.
In order to measure performance, DevMagic were relying on a series of manual reports to obtain, visualise and share data. The team found this to be a labour intensive and time-consuming process, so began to seek a platform that could provide real-time performance metrics , enable a customer-centric focus, and expose opportunities for improvement.
It was during a webinar on ‘Automating fit-for-purpose analysis in software delivery’ featuring Flomatika’s CPTO Marcio Siete, as part of Kanban Brazil 2021 that the team serendipitously discovered Flomatika. Already deep in their search, Marcio’s key messages of being ‘fit for purpose’ and how platforms like Flomatika can help organisations achieve this goal particularly resonated with the team.
Seasoned trailblazers, DevMagic quickly reached out for a demo and, following a short trial, fully incorporated Flomatika into their operating cadence.
A one hour implementation workshop was all it took to connect and ingest DevMagic’s data, but also acted as a training workshop for the team on how to configure, administer and get the most out of the platform. Many of the key concepts around Flomatika were covered by moving through the configuration wizard together, allowing those involved in the session to share this knowledge with the wider team.
DevMagic’s Service Delivery Manager, Luiz Reis, has incorporated the use of Flomatika into their regular operating cycle. He has introduced 3 regular events and soon to be a fourth.
Weekly Service Delivery Review
At the start of every week, the Service Delivery Manager conducts service delivery review sessions with each team. Focusing particularly on customer satisfaction, Flomatika is used to allow each team’s performance metrics to be measured against their service category’s service level expectations (SLEs) to determine whether they are being met and how they are trending over time.
Similar SLE analysis is also conducted by work item type (feature, bug, enhancement, tech debt and risk), as is analysis around classes of service (expedite vs normal), by which they determine the process improvements, especially for expedites, should be undertaken in the next development cycle.
Each week they pinpoint individual items that exceeded their SLEs to ask how to better approach them, while also identifying the items that were well under their SLEs and asking how they can be replicated.
They also review current items in WIP, applying learnings from the previous week to detect and address any emerging patterns. They identify items that have already gone over, or are about to reach, their SLE and ask how they should be managed.
Finally, they examine their burn ups and burn downs to check how they are tracking towards their overall goals and objectives.
Flow Lens: Shows completed items for the selected time period by Demand Type, to what degree they are meeting SLEs, and their lead time distribution.
Flow Items: Shows each individual completed item, along with all their information, such as Lead time % of SLE.
Monthly Service Delivery Review
Similar service delivery review sessions are also run monthly with each team using Flomatika, however these are less tactical and more strategic. Scrutinising a longer time period, the focus shifts from data analysis to reflection.
There is also a monthly comparison of services between all teams and an exchange of knowledge about processes and development routines, highlighting and utilising each teams’ unique expertise.
Throughput widget: Shows throughput for the selected time period (including average per week)
Productivity widget: Shows productivity per week using ‘natural’ language such as ‘Good’, ‘Excellent’, or ‘Poor’, rather than using a number that has no meaning when looked at in isolation.
Specific metrics within Flomatika are reviewed as part of the team’s daily sync to help make better on-the-spot decisions.
This includes identifying any items belonging to the expedite class of service, to ensure they receive sufficient attention.
Next, they review their WIP count to see if they have surpassed their limits, asking what can be done if they have.
To finish, they review all items that have exceeded or are about to reach their SLE and determine if any action can be taken.
WIP Widgets: Shows the number of items currently in WIP, as well as the average across assignees. Also shows how long on average items have been in WIP for.
Flow Lens: Shows WIP Age distribution by demand, as well as by individual flow item type.
Flow Items: Shows each of the individual flow items in WIP, along with their associated information, such as WIP Age in % of SLE.
‘Post Mortem’ or Retro
DevMagic will soon introduce a fourth event that incorporates the insights gained from Flomatika. Called a ‘post-mortem’, it is essentially an end-of-sprint retrospective, with a focus on metrics analysis.
Post-mortems enable reviews of data across the sprint and analysis of the various service types and to what degree they have met their SLEs, as well as their lead time distribution. They also facilitate drilling down by individual items to see how they met their SLEs and reveal how burndown looks across the sprint, as well as historical throughput.
With insights gained, DevMagic are now able to summarise the difficulties encountered, determine which demands need more time, and identify opportunities for improvement and critical future actions.
Burn Down and Burn Up charts: Shows remaining work across the time box.
Lead Time Scatter Plots and Histograms: Scatter plot shows lead time over time, whilst the histogram shows the ‘frequency’ of lead time.
Adopting Flomatika has allowed the DevMagic team to be more focused on delivering value to the users, rather than trying to figure out how to capture their metrics.
To this end they benefited from greater visibility into the flow of their work, and where the problems were. Items that were close to compromising their SLEs were highlighted.
A considerable amount of manual work was eliminated for the Service Delivery Manager, allowing them to spend more time discussing the insights gained and the actions to be taken.
Not only did increased visibility enable the teams to focus on delivery performance, but the benefits extended to internal capability uplift as well.
Flomatika also enabled each individual to assess their own performance and pinpoint their areas of improvement, which supported discussion in professional development. Flomatika was also able to identify individuals with higher proficiency in specific technologies, thereby creating new leaders within the organisation.
Ultimately, Flomatika is helping DevMagic build a culture of continuous improvement, by constantly introducing new approaches to their delivery practices, consistently lowering their lead times, whilst still maintaining quality.
DevMagic’s initial focus on leveraging the Flomatika platform has been around lead time, and ensuring that their teams are comfortable working within the SLEs of the respective service requests. They will soon shift their focus into other aspects of their delivery to fully leverage the Flomatika platform and take them to the next level to ensure they are always a few steps ahead of their competitors.
- Massive lead time reductions across the board. On average a reduction of ⬇️ 41.37%. Through the visibility gain via Flomatia, they were able to focus on improvements in specific facets:
- In the first half of the year (2022), the median of lead time for bugs dropped from 12 days to just 4 days - representing a drop of ⬇️66.66%
- In a 7-month period were able to reduce the lead time working with a new technology from 21 days to 12 days - a reduction of ⬇️42.85%
- Reduced upstream discovery time by ⬇️30.76%, via a process of identifying common root causes, which was facilitated by the visibility provided by Flomatika
- Lead Time predictability ⬆️ improved across all teams, across all service categories, once again enabled by the visibility gained from Flomatika
- Increased productivity in a quarter by ⬆️12.65% (an increase in throughput)
- Identified a common root cause in items with high SLEs, which resulted in the creation of a new technical review process that reduced the likelihood of these items surpassing their SLEs
- Since the adoption of Flomatika, a number of teams have been able to consistently deliver within SLE for an extended period of time. ⬆️Increase in delivering within SLEs results in happier customers! Teams were also able to share their knowledge and experience (driven by data within Flomatika) with each other to apply learnings.