In the last article (Steele, 2020), we examined the process of finding and assessing available evidence from scientific literature and compiling it into a Knowledge Summary to aid clinical decisions. This communication will concentrate on obtaining and evaluating the effects of using evidence-based techniques to support professional or commercial needs. In order to show any value in something, an understanding of the requirements it may fulfil is essential. It is also necessary to consider why an evidence search or analysis is needed and to define what its purpose might be in order to get a clear idea of what expectations are to be gained. Only when this has been determined, can the user begin to monitor outcomes from their case management and any differences from the original situation be assessed. This can be linked back to the initial requirements and either personal, professional or commercial value be calculated.
There are several online sites available to supply information on how to search for evidence and that provide Knowledge Summaries already completed, in order to save time and effort to the busy practitioner and allow them to concentrate on dealing with their case or clinical question (Centre for Evidence-Based Veterinary Medicine, 2020; RCVS, 2020). However, there are very few sites available to show the effect that the search might have had on the user. In the future, it may be interesting to consider central sites or databases where an individual or group may be able to upload their experience of using evidence-based medicine connected to the outcomes of their cases. It seems this is the intention of the Nottingham University Centre for Evidence-Based Veterinary Medicine as well as the RCVS in their recent addition of encouraging the recording of outcomes in personal Continuing Professional Development (CPD) documentation. However, this is not yet in an accessible state for the profession to benefit from those results. This would go a long way towards helping the practitioner to be able to see a professional or commercial benefit in the usage of a ‘best evidence’ approach.
Defining value and evidence
The word value is very difficult to define, as whatever an individual gets from an object or process, is extremely variable and dependent on that individual's wants or needs at the time of asking. Therefore, it is useful to pass this on to the shareholders staking a claim on the results of the evidence review, namely the practitioner, practice, customer and ultimately, the animals. Asking the right question at the beginning is crucial to be able to determine:
- Any differences between the start of the process and the end
- What that means in terms of meeting these requirements commercially or professionally: what measurable gains can be made, and over how long?
The most important part of value determination is to ask the user, customer or client exactly what they are expecting from the process and to listen carefully to that answer. To be able to capture the maximum value from the resulting intervention chosen by your evidence-based review, it will be necessary to measure the outcome that the client was expecting: often, in the author's experience, this is not related to commercial gain or a direct ‘cure’ of the condition. It is related to what benefits can be obtained from a situation that is alleviated from your intervention: perhaps this is less time spent dealing with debilitated animals and more time spent with the owner's family instead. So, by all means, measure disease prevalence and its reduction but also ensure that you measure what the clients' needs were at the start (e.g. time spent with their family before and after).
The second element in determining value from evidence is to define which evidence is to be evaluated. Are we looking to increase our personal knowledge, increase the capabilities and therefore professional services of the practice or provide an intervention to a case which aims to reduce the risk of an adverse outcome? The result of asking this question can assist the user in choosing the right measurable data in order to evaluate the effect of the evidence search. So disease outcomes, antimicrobial use or improvements in fertility or growth parameters are obvious indicators of commercial gains, but to capture the maximum from using the ‘best evidence’ approach, we need to also consider reduced labour by following more efficient procedures and perhaps client opinion surveys, practice group surveys or digging deep into software databases from farm records across the practice or even national data.
Using evidence-based veterinary medicine (EBVM) in practice
Although the last article (Steele, 2020) showed the reader how to search the evidence databases and to appraise the resulting scientific papers, it did not consider a full list of uses of the evidence or include the benefits of using data that are available to the practitioner within their own practice.
An internet search for journals or even a web-based database of evidence-based knowledge summaries can be obtained by even a quick enquiry to a search engine. However, these are not the only sources of information that the practitioner can use. Within most farm animal practices, the practice software system may be able to provide compilations of data from farm records under their care, which may be possible to collect: for instance, you might be able to ask the software to provide all mastitis records from the practice's database and then filter this to ‘new cases only’, ‘treated with product X’ or ‘parity 1’ over a specified time, to answer a question on efficacy. Some practice software can even provide anonymised, local or national data within their server to get a broader and more reliable result. Be aware though, that data are only as reliable as their input (Overton, 2006; McCarthy and Overton, 2018), and as sources become wider, there will be more scope for misdiagnosis and error but the hope is that this effect may become more diluted as the data volume increases. Within food animal production, there are quite extensive sources of data available commercially such as the Agriculture and Horticulture Development Board, UK website for poultry, pig, dairy and beef trends and knowledge sources or commercial subscriptions such as National Milk Recording (NMR) where data are stored and available for the subscriber. Although the intentions of these sites are to share data, they are restricted to sub-scribing customers and the contributors to the complete dataset are therefore limited. There are some organisations that are seeking to amalgamate data across private, commercial databases (e.g. Dairy Data Warehouse), but they have huge challenges in unifying data: mostly this is in aligning definitions to parameters, for example, pregnancy rates: there are many ways to record pregnancy and fertility and to ensure that the user gets an accurate parameter that is useful to them is extremely difficult to achieve. As time goes on, this will improve and the hope is that the practitioner will be able to get reliable, national records in order to make more informed clinical decisions in the future.
Showing the value
The first step in being able to show the value that an evidence search has made to your clinical decision, is to determine what you are going to do with the answer. This links back to the section on defining value and the determination of what measurable gains you can make from the result. Some suggestions of the way in which practitioners are using evidence searches come from a quick search of the internet: one veterinary hospital is publishing their reviews and the results of their evidence-based discussion group (Banfield Pet Hospital, 2020). In this way, they are increasing the capabilities of the practice staff and sharing their knowledge base. Some individuals may need the evidence search to simply provide them with more hours of CPD, to aid their own knowledge base, and other practitioners use evidence searches to provide advice to customers or colleagues on standard operating procedures. Within the animal health industry, evidence searches can help to decide on the direction of clinical trial design, which innovation ideas might or might not work or to provide reliable marketing material to their customers.
Whatever the intended use of an evidence search, it is valuable to consider inserting it into a consistent process. ‘Lean’ principles are popular approaches in many industries today. Developed by Toyota in the 1950s, they focus on reducing ‘waste’ in a process, making the resulting outcomes more efficient: in our case, waste would be inefficiencies resulting from diseases, such as treatment times, lost animals, increased costs and subsequent carbon footprint increases. Lean processes can be a very helpful way to look at clinical case management process (Machado and Leitner, 2010) (Figure 1). First, build a Problem Statement, which is related to the initial challenge: this should be SMART (specific, measurable, achievable, relevant, timely) and clearly state the current situation and the required (target) situation, linked to a value parameter. An example may be to reduce new cases of clinical mastitis from x% to y% in 3 months, saving £XX,000 at US$444/case (Rollin et al, 2015). The evidence to support the investigation can include research into costs of clinical mastitis cases, data from the farm records and comparisons from national databases to help to decide what is achievable and realistic in the timeframe. This must be considered along with a visit and careful examination of cases, culture results and milking routines etc. From the results, it may be prudent to re-consider the original problem statement, as the data may point to another focus (or they may not). From this, a solid goal can be prepared and the practitioner can proceed into root cause analysis (Iedema et al, 2006). Once this is determined, a clear target situation can be built (to continue the example of mastitis, this may be to re-calibrate the dynamic vacuum pressures in the parlour, in order to reduce cases and restore profitability in 3 months). Countermeasures must be determined, such as servicing the parlour, ensuring correct and accurate mastitis records are taken, regular analysis of results and a project devised to carry this out, with an accountable ‘who does what, by when’, mapped by day and week over the 3 month course of the project. Follow-up would include analysing all mastitis, yield, cell count, parlour timings, involuntary culling, antimicrobial usage data and any other parameter helpful to capture the full value, including the labourer's time spent caring for mastitic cows. The results would be precise and fully evidence-based, having included careful research into the scientific literature, examination of farm and national data, and the outcomes would be communicated and aligned to all involved in the project.
This would not be the end of the procedure. In order to ensure that the maximum value is obtained, the outcome should be shared in a repository where other members of the profession can benefit: not only to see what value can be achieved when practicing evidence-based medicine, but also to prevent re-invention of the wheel — each time a similar case occurs, others can apply the process and save time and effort doing the same.
Conclusion
Understanding the value of using evidence-based medicine techniques involves clearly defining the full benefits that can be obtained from possible results. Much of this is determined by questioning the stakeholders (practitioners, practice and farm staff) at the start of the process to find out what they require from meeting their specific challenge. If this is clear and aligned within the stakeholder group, assessment of the results can be more easily established. Evidence can come in the form of scientific literature review as well as the collection of as much animal or environmental data as possible, related to the case in question. Using a defined and consistent problem-solving process can have a hugely beneficial effect on the success of a project. Evaluation of the differences in the situations at the start and end of the case process, including the potential lifestyle benefits to the stakeholders (and the animals), can be made in order to capture the full value.
Sharing the results of evidence-based case management between the practice, other professionals and on more accessible, online resources in the future could allow advisors to be not only better informed but more efficient to the industries they serve.
KEY POINTS
- Showing value from evidence-based problem solving must always start with LISTENING to the need of the client, not your opinion of what needs to be fixed.
- Establishing potential value must be based on understanding the evidence collected from the problem process.
- Evidence can come in the form of data, a thorough walk through the process and literature appraisal.
- Projects are more likely to succeed if they follow a defined path, based on an agreed root cause of the original problem.