Psychology Wiki

Talk:Criteria for the assessment of clinical studies

34,135pages on
this wiki

Back to page

A beginningEdit

I need to come back with that citation and tidy up generally-- sorry, i don't know how to turn that banner into regular format.Jean Mercer 15:00, 4 October 2007 (UTC)

Thats a good start Jean thank you. The banner shows when the text is not left justified. It happens when you copy over text where the paragraphs are indented. You just have to remove the blank spaces. I will do it now no problem Dr Joe Kiff 19:40, 4 October 2007 (UTC)

Joe, what's the story with the changes-- did you think it was too OR? I was trying to show how thinking on this issue has changed and is continuing to change. Jean Mercer 21:36, 4 October 2007 (UTC)

Jean do you want all the citations turned into little numbers or are some of them to be left as names? Fainites 21:09, 4 October 2007 (UTC)

Little numbers is great. Thank you--- I hate doing that part.Jean Mercer 21:36, 4 October 2007 (UTC)

Little numbers as promised. Don't forget the missing citation. Fainites 22:27, 4 October 2007 (UTC)

Do you have a full copy of this one Jean as I only have the abstract [1] Thanks. Fainites 22:32, 4 October 2007 (UTC)

The Kaufman link doesn't work. I've got another one here. Can you just check it before I stick it in.[2] Interestingly the Kaufman 'Best Practice Approach' worked its way through candidates and the three best who achieved category 1 (like Saunders category 1) they called 'Evidence Based' with capital letters! Fainites 22:41, 4 October 2007 (UTC)

That Kaufman seems to be right-- thanks. Sorry, i don't have the Chaffin & Friedrich, which sounds good. It must be one of the last things Bill Friedrich wrote before he dies-- what a loss.Jean Mercer 22:43, 4 October 2007 (UTC)

Hang on - I'll see if I can get it by another route. Can I add a little more to Kaufman? To be a candidate at all requires at least one randomized clinical trial. Then they set out their 6 levels as follows: 1. Well supported, efficacious treatment 2. Supported and probably efficacious treatment 3. Supported and acceptable treatment 4. Promising and acceptable treatment 5. Innovative and novel 6. Experimental or concerning treatment.

This looks very much like Saunders, if not identical as to headings, and indeed he was part of the project. However, Saunders guidelines only requires randomized clinical trials at level 1, and then he requires two of them, whereas he doesn't require one at all for any of the other levels. Is this right?

Kaufmans minimum criteria to even take part; "TO BE CONSIDERED A CANDIDATE FOR BEST PRACTICE, A TREATMENT PROTOCOL HAD TO MEET THE FOLLOWING CRITERIA CONCERNING ITS CLINICAL UTILITY: 1. The treatment has a sound theoretical basis in generally accepted psychological principles indicating that it would be effective in treating at least some problems known to be outcomes of child abuse. 2. The treatment is generally accepted in clinical practice as appropriate for use with abused children, their parents, and/or their families. Candidate practices were sought that met several minimum criteria. 3. A substantial clinical-anecdotal literature exists indicating the treatment’s value with abused children, their parents, and/or their families from a variety of cultural and ethnic backgrounds. 4. There is no clinical or empirical evidence, or theoretical basis indicating that the treatment constitutes a substantial risk of harm to those receiving it, compared to its likely benefits. 5. The treatment has at least one randomized, controlled treatment outcome study indicating its efficacy with abused children and/or their families. 6. If multiple treatment outcome studies have been conducted, the overall weight of evidence supports the efficacy of the treatment. IN ADDITION TO THESE CRITERIA SUPPORTING THE EFFICACY OF THE TREATMENT, THE FOLLOWING CRITERIA ABOUT ITS TRANSPORTABILITY TO COMMON CLINICAL SETTINGS HAD TO BE MET: 7. The treatment has a book, manual, or other writings available to clinical professionals that specifies the components of the treatment protocol and describes how to conduct it. 8. The treatment can be delivered in common service delivery settings serving abused children and their families with a reasonable degree of treatment fidelity. 9. The treatment can be delivered by typical mental health professionals who have received a reasonable level of training and supervision in its use. Once these candidates were identified, the project focused on reviewing the support for each one, understanding their weaknesses in clinical or empirical support, and building consensus among the participants."Fainites 23:01, 4 October 2007 (UTC)

We seem to have Saunders in twice - once under 'Saunders' and once under 'systems from the child abuse field'. Fainites 23:08, 4 October 2007 (UTC)

I've found a copy of that Chaffin and Friedrich paper on EBT - only one page missing! I'll e-mail it too you Jean.Fainites 23:29, 4 October 2007 (UTC)

Jean. What does OR mean? I was mainly just trying to highlight the sections so its easier to find details. Looks as though its turning out well Dr Joe Kiff 06:31, 5 October 2007 (UTC)

I feel we need to change the title to make the focus clearer. I have started a page up in the statistics section entitled Evaluating scientific evidence . I suggest we call this article "Criteria for evaluating the evidence base of a therapy" to distinguish it from how to evaluate a scientific paper. Any other suggestions. We could put something at the top of this article to make it clearer that it is about evaluating a body of evidence and assessing therapies against different scales reflecting degrees of scientific rigour. Dr Joe Kiff 06:31, 5 October 2007 (UTC)

I think the last section deserves an article of its own. Entitled The limits of the scientific method in clinical psychology Dr Joe Kiff 06:58, 5 October 2007 (UTC)

Currently the last section is a copy paste from the Wiki EBT article, only without the 'disputed' tag. It needs a bit of work to make it more suitable for a psychology Wikia rather than medicine in general, but I think there could be a summary of the controversy linked to its own article. For issues on evidence base in the psych fields good sources are Chaffin/Friedrich, Gambrill etc, as well as the various people already piling in with suggested methods. Fainites 22:17, 6 October 2007 (UTC)


Joe--OR is original research-- i.e., you can't cite a source.

Its a Wikipedia term WP:OR Anything that isn't sourced (or flaming obvious and not needing a source) is verboten on Wiki. Paraphrases and summaries are fine - provided they're accurate, but you can't put your own views in. I think we need to stick to the same rules here, except, if you think the site as such is in a position to express colleective wisdom, that would be fine, as long as its clear that that is what it is. eg I suppose after saying Kaufman says this and P & G say that you could say 'the authors of this article believe' or 'are of the opinion that..' but it has to be clear I think. Fainites 16:50, 5 October 2007 (UTC)
I take a somewhat different view I actually think the strength of the wiki is that after consolidating and clarifying knowledge in a field and developing a consensual critique it can provide a context for original thinking and for discussing original research. I think the key is that they should be clearly marked out seperately from the the pages reviewing published work. So here if Jean would like to share her ideas of combining the scales that would go into a seperate section or on a seperate page. I see it more as an active tool for thinking that will be capable of moving the discipline forward more quickly. We could wait 18 months for it to be published in a journal we cant get hold of easily, copyrighted so we have to pay exhorbitant fees to see it and then have sporadic discussions about it in letter columns in the same journals with the same restrictions, and in conferences which most of us can't attend. Here all this can be more immediate and I think more fruitful. Look at the way this article has been thrown up in a matter of days, I know Jean has done a lot of the spadework already but I assume its already more comprehensive than the original article with more benefits to come Dr Joe Kiff 05:24, 6 October 2007 (UTC)
Oh I don't disagree with that - provided its always clear which is which. IMHO the basic articles providing info. ought to be very clear and careful about sources, always attribute everything, accurate representation, notable sources, distinctions between primary and secondary sources and so on and so on in order to make them trustworthy if that is at all possible. Then we can have sections within an article perhaps or articles for eg commentary or synthesis by the articles authors if there is a collective view, or eg something by a particular author like Jean. Perhaps there could be specific Forums linked for hot topics as opposed to the editors talkpage.Fainites 17:44, 6 October 2007 (UTC)
Clarity is the key. I like Forums but we will have to wait for a more active community to develop as they need intellectual momentum generated by more than arguing over the content of articles!! Dr Joe Kiff 16:52, 7 October 2007 (UTC)
OK. Good, solid, reliable, sourced articles first, then its a spring board for moving on. No point discussing the rights and wrongs if people don't have a decent overview of a particular area, current research, mainstream thinking and sensible controversies. Fainites 18:49, 7 October 2007 (UTC)

F., I said Kaufman only accepts RCTs... does it need to be more specific? Do you want to add those categories? Are you sure you aren't looking at the ones from Saunders et al.? I don't have the Kaufman in front of me right now, but the taxonomy you've put on this page doesn't sound like Kaufman to me.Jean Mercer 12:26, 5 October 2007 (UTC)

Well the thing I put on this page comes from the link to Kaufman. Have a look. Maybe I'm missing something - but the list is identical to Saunders except they only accept randomised designs whereas Saunders only requires randomised designs for category 1. Please look at the Kaufman link and see if you can work it out. Maybe I read it hastily. Fainites 16:32, 5 October 2007 (UTC)

I think I've got it now - Kaufman and his lot use the criteria I've put above for candidates for Best Practice. They started off looking at the 23 or 4 treatments Saunders had looked at in 2003 under Saunders rules. Only one made category 1 in 2003. Kaufman looked at the ones in categories 1, 2 and 3 so presumably some of them must have done some more research between the Saunders assessment and the Kaufman assessment in order to qualify for the kaufman protocol, and indeed they imply as much and say more is on its way. Fainites 16:45, 5 October 2007 (UTC)

I think what I'll do is set out the full criteria for each system in footnotes, unless anyone thinks they ought to go in the article. Fainites 17:46, 6 October 2007 (UTC)

I'm setting them up in the footnotes for now but then I think each should go on a separate page with a link. Fainites 10:59, 7 October 2007 (UTC)

I agree that there ought to be a page for each as presumably there are commentaries and references for each.

I am thinking about setting up a template so that research in any approach could be evaluated by each set of criteria. But I think there are too many systems and we might work up a proposal for a joint sytem taking the best features from whats available. Was that the plan of Jeans article? Would this cause problems, would Jean like to hold off until she is published? At least it may be worth listing the features we might include as a start Dr Joe Kiff 16:52, 7 October 2007 (UTC)

I think that Fainites suggestion of spelling out the criteria for each system is a good one. I'd suggest maybe doing that in this article so that a unified common presentation is developed and then putting the material into sub-pages if needed (if the article becomes too long and cumbersome) Or, we could just list the criteria here and then use sub-articles to provide detailed notes and commentary. This way a reader can see what systems are available, evaluate each one and then use the system most relevant to that researcher/clinician/reader's requirements. What you all think? Cheers, Dr. Becker-Weidman Talk 16:57, 7 October 2007 (UTC)

Yes now I've started putting them in the footnotes it looks a little messy. We can try putting them in the body of the article and see how it looks, but I think clearly linked separate pages is probably the answer. We can keep the brief descriptions, general round up, outline of arguments, controversy here and then have all the precise details on the pages. It would be nice to have a sort of spreadsheet that had them all side by side by I suspect that would be impractical. Fainites 18:45, 7 October 2007 (UTC)

Oh I see you've already done that for Saunders. I'll shove in the others. Fainites 18:51, 7 October 2007 (UTC)

Nicely done. cheers. Dr. Becker-Weidman Talk 23:21, 7 October 2007 (UTC)

Kaufman vs. SaundersEdit

What happened with Kaufman is that they started their selection by making a list of interventions classed 3 or better on the Saunders system. Then, reading new research, etc., they chose from that group the ones that met their criteria for best practice, which were similar to the highest category for Saunders but had other requirements as well. In the end there were only the two categories, best practice and no-best-practice. An interesting point here is that however an intervention is classified at a given time, it may be changed in category by new work. This is quite likely to happen if a researcher has been subject to an "allegiance effect" and an independent replication is attempted.Jean Mercer 01:05, 8 October 2007 (UTC)

Yes - I worked it out above! I misread it initially - I thought they'd used similar words for two different systams - very confusing. Mind you, a number of systems do do that, like 'Level 1 and 2', or 'A, B and C' etc. Fainites 09:18, 8 October 2007 (UTC)

Chambless and HollonEdit

i've started adding the details of this one. I'm doing it in some detail as its the APA one and therefore pretty important I would think. Hope to finish soonish. Fainites 10:03, 9 October 2007 (UTC)

Edits to SaundersEdit

I edited this section to read in a manner parallel to the other approaches; keeping a consistent style and approach here.. Dr. Becker-Weidman Talk 16:10, 30 October 2007 (UTC)

Around Wikia's network

Random Wiki