Approaches to defining Basic vs Advanced Features… Manufacturers, Existing Definitions or Other Conceptualisations?

Continuing from my previous post and the extended response from Christina Silver at

  1. On what grounds is the basic vs advanced rejected? Is there alternative evidence to assert this might not be such an easy rejection to defend. (Spoiler: Lots IMHO)

Now Christina has, most flatteringly, responded to my initial blog post with a very extended consideration in response. This enables me to engage in dialogue with soemthing much MUCh more considered and nuanced than a tweet – which is great. In her response she argues that:

Distinguishing between ‘basic’ and ‘advanced’ features implies that when learning a CAQDAS package it makes sense to first learn the ‘basic’ features and only later move on to learning the ‘advanced’ features. In developing an instructional design this begs the question of which features are ‘basic’ and which are ‘advanced’, in order to know which features are taught first and which later. We remain to be convinced how this distinction can meaningfully be made. What criteria are used to decide which features are ‘basic’ or ‘advanced’? Is it that some features are easier to use than others? Or that some features are more commonly used than others? Or that some features are used earlier in a project than others? I’m interested to hear what others criteria are in this regard.   We believe that attempting to distinguish between ‘basic’ and ‘advanced’ features is unhelpful. – See more at: http://www.fivelevelqda.com/article/10640-there-are-no-basic-or-advanced-caqdas-tools-but-straightforward-and-sophisticated-uses-of-tools-appropriate-for-different-tasks#sthash.OIBTaEEG.dpuf

Now, I can really see the point and purpose of this approach, but also wonder if there is some merit in exploring and contesting it.

What criteria are used to decide which features are ‘basic’ or ‘advanced’?

Option 1 – using Manufacturers’ product differentiation

One way of defining this would be to draw on the way packages are marketed, developed and positioned. And the manufacturers provide plenty of text and charts and details to do just this. WHY? Well these classifications exist, they are in play, they are acting as differentiators between packages. They will be guiding people and positioning options as well as costs.

From a teaching perspective I can also see a huge benefit – stripped down software with fewer options is just far FAR less daunting! i have seen students looking slightly terrified of the complexity and option of NVivo or ATLAS.ti really light up when F4 analyse is introduced.

F4 analyse is part of the new generation of “QDA Lite” packages. These include the EXCELLENT F4 analyse as well as the quirky, touch-oriented QUIRKOS. Joining this grouping are also the cut-down versions of “full featured” packages: NVivo Starter , MaxQDA Base. Potenitally we could also include tablet-versions of key packages such as the ATLAS.ti  app and  MaxQDA App .

Looking across these we could come up with a list of common features that would provide an empirically based list of “features that are included in basic versions of QDA software” and thus achieve a working definition of “basic features”.

The list from F4 Analyse seems pretty good to work from:

  • Write memos, code contents
  • Display and filter quotations
  • Develop a hierarchical code system
  • Description and differentiation of codes
  • Distribution of code frequencies
  • Export the results

My suggestion here is that these packages DO position some technologies as simple and others as advanced – seeking to erase rather than reposition that difference could therefore be less productive even if it is theoretically justified.

Option 2: Established definitions

Alternatively we could go back to older existing and established definitions e.g. those proposed by the CAQDAS networking project :

Definition

We use the term ‘CAQDAS’ to refer to software packages which include too ls designed to facilitate a qualitative
approach to qualitative data. Qualitative data includes texts, graphics, audio or video . CAQDAS packages may
also enable the incorporation of quantitative (numeric) data and/or include tools for taking quantitative
approaches to qualitative data. However, they must directly handle at least one type of qualitative data and
include some – but not necessarily all – of the following tools for handling and analysing it:

  • Content searching tools
  • Linking tools
  • Coding tools
  • Query tools
  • Writing and annotation tools
  • Mapping or networking tools

The combination of tools within CAQDAS packages varies, with many providing additional options to those listed here. The relative sophistication and ease of use also varies and we aim to uncover some of these differences in
our comparative reviews

So here again we have a list of tools that could be considered to be “basic” with the additional criteria of “relative sophistication” and “ease of use” giving dimensions for considering those criteria.

But – does that do anything?

Option 3 – (A bit of a “thought in progress… “)Conceptualising Affordances

Affordances are both an easy shorthand and a contested term (see Oliver, 2005) but one that rains both a common-sense understanding of “what’s easy to do” or maybe – with a more interactionist or even ANTy sensibility of non-human agency “what actions are invited” – that whilst it may lack the sort of theoretical purity or precision that may be desired remains a useful concept.

How then could “the affordances of CAQDAS” be explored systematically, empirically and meaningfully?

Thompson and Adams (2011, 2013, 2016) propose phenomenological enquiry as providing a basis. Within this there are opportunities to record user experience at particular junctures – moments of disruption and change being obvious ones. So for me encountering ATLAS.ti 8 presents an opportunity to look at the interaction of the software with my expectations and ideas and desires to achieve certain outcomes. Adapting my practices to a new environment creates an encounter between the familiar and the strange – between the known and the unknown.

However is there a way to bring alternative ideas and approaches – perhaps even those which are normally regarded as oppositional or incommensurable with such a reflexive self-as-object-and-subject mode of enquiry? Could “affordances” be (dare I say it?) quantified? Or at least some measures be proposed to support assertions. For example if an action is ever-present in the interface or only takes one click to achieve could that be regarded as a measure of ease – an indicator of affordance?

Could counting the steps required add to an investigation of the tacit knowledge and/or prior experience and/or comparable and parallel experience that is drawn on? Or would it merely fudge it and dilute it all?

My sense is that counts such as this, supplemented by screen shots could provide a twin function – that is the function of trying to map and uncover the easiest path or the fewest steps to achieving a desired outcome which will not only provide a sense or indication of simplicity/affordance vs complexity/un-afforded* (Hmmm – what is the opposite of an affordance? If there isn’t one doesn’t that challenge it’s over-use?) action but also the basis for teaching and action based on that research – to show and teach and support ways around the easy routes written into software that configure the user.

Drawing this together

This is part of my consideration of simplicity vs complexity and how this distributes agency when working with complex technologies for qualitative analysis. I’m not convinced that the erasing of simplicity vs complexity is the right way to approach this. here I’ve tried to set out some ideas and existing approaches which are already circulating and propose some ideas around the influence these have and my experiences too.

This is in part to anticipate lines of argument or proposals  about something being simple, basic or easy which that have some demonstrable grounding.

But where is this going – well there’s two aspects to my thinking:

  • one aspect is about complexity in practice: how do software packages shape our practices and make some things very visible and very simple to achieve? I’ve started sketching this out with the affordances bit here but there’s something more to it.  I do believe this can be empirically considered and assessed in terms of visibility and complexity in local practice – whether that is the number of clicks to get to something or the number of options available to customise a feature. It can also be considered more generally in terms of consideration of the shaping of method and patterns of use and non-use and how certain approaches to qualitative research become reinforced whilst others become marginalised from a software supported paradigm.
  • the other is a more comprehensive argument about the challenge and problems and potential for missed opportunities. My concern here is if and how the transformative potential of tools are not realised if and when they are made subservient to strategies based on older ways of working from when such tools were not available. The outcome of that is that the potential of tools would be something important to foreground and explore as these can (and I would argue should) lead to new strategies that were simply not possible before… And that’s the topic of my next post. 

So this was a first step to respond to one aspect of the argument Christina and Nicholas advance. Their approach is one one which I think has huge merit, however, as with anything of merit for teaching and practice I also believe there is a value in contesting it in order to explore, deepen and enhance it and anticipate lines of critique as well as developing responses to support its use, implementation and adaptation.

 

References

Adams, C., & Thompson, T. L. (2016). Researching a Posthuman World Palgrave Macmillan UK.

Preview at https://books.google.co.uk/books?id=RdGGDQAAQBAJ&lpg=PP1&pg=PP1#v=onepage&q&f=false

Adams, C. A., & Thompson, T. L. (2011). Interviewing objects: Including educational technologies as qualitative research participants. International Journal of Qualitative Studies in Education, 24(6), 733-750.

Oliver M. (2005) The Problem with Affordance. E-Learning 2: 402-413.  DOI:10.2304/elea.2005.2.4.402

Thompson TL and Adams C. (2013) Speaking with things: encoded researchers, social data, and other posthuman concoctions. Distinktion: Scandinavian Journal of Social Theory 14: 342-361.

E-Print available at http://www.storre.stir.ac.uk/handle/1893/18508#.WRsL51MrKV4

Basic vs advanced CAQDAS features?

Part one of a series of posts in dialogue with Christina.

There are no basic or advanced #CAQDAS features, but straightforward or more sophisticated uses of tools appropriate for different tasks

— Christina Silver (@Christina_QDAS) April 27, 2017

This tweet got me thinking a LOT about the ideas it  – it’s a tweet so it’s trying to distill a complex argument down into a pithy soundbite. However something about it doesn’t sit quite right with me. This blog post is an attempt to start working out some of those questions and hopefully do so in a space with sufficient space (rather than twitter character limits) to engage in dialogue but also work out the issues at some length.

I want to try and break it down into it’s key aspects the engage with each:

There are no basic or advanced #CAQDAS features

CAQDAS = Computer Assisted Qualitative Data Analysis Software

Basic vs Advanced features = not only a false dichotomy but something that doesn’t exist

Instead there’s a new dichotomy proposed of:

Straightforward vs sophisticated uses of tools.

And the straightforwardness or sophistication is to be judged in terms of their “appropriateness for different tasks”.

My key questions therefore are:

  1. On what grounds is the basic vs advanced rejected? Is there alternative evidence to assert this might not be such an easy rejection to defend. (Spoiler: Lots IMHO)
  2. The more complex exploration of how would a judgement of appropriateness be based for considering if you are doing “straightforward” or “more sophisticated” use of tools, and how would those tasks be determined in a way that to me at least reads as being independent of, preceding or separable from the tools?

Fundamentally, I see this as a question of the distribution of agency between

  1. manufacturers and designers of tools,
  2. the tools,
  3. the tasks that can be done, and
  4. the users.

I interpret this formulation as being one that sees or proposes that the agency is (or should be) primarily with the users. Which I further interpret as proposing a new way to (re)configure the user – to draw on Grint and Woolgar’s (1997) conceptualisations.

RESPONSES:

I’m VERY pleased to say that Christina has responded to this post to expand those ideas substantially over at http://www.fivelevelqda.com/article/10640-there-are-no-basic-or-advanced-caqdas-tools-but-straightforward-and-sophisticated-uses-of-tools-appropriate-for-different-tasks in response to this post. So I shall compose further responses in other linked posts.

CONTINUING THE DISCUSSION:

On considering and defining basic vs advanced tools – which is pretty minimal but proposes possibel criteria.

And a much more extended consideration of the distribution of agency and relationships between tools, potentials, strategies and tactics. 

Current Reading – Engaging with Content Analysis and a different notion of “coding”

I’m currently reading these two books:

Krippendorff, K. (2013). Content analysis: An introduction to its methodology (3rd Edition). Sage.
https://uk.sagepub.com/en-gb/eur/content-analysis/book234903

(I note someone’s put a full copy of the second edition up on academia.edu if you google for it… But you didn’t read that here 😉

This is a VERY readable introduction to content analysis which is really interesting and has a great section of computer support including extended considerations of the use of CAQDAS packages such as ATLAS.ti, NVivo and the more content-analysis oriented QDA Miner / WordSTST combo.

I’m now starting:

Leetaru, K. (2012). Data mining methods for the content analyst: An introduction to the computational analysis of content. Routledge.
https://www.routledge.com/Data-Mining-Methods-for-the-Content-Analyst-An-Introduction-to-the-Computational/Leetaru/p/book/9780415895149

Content Analysis and Coding vs Inductive impressions.

Will need to turn this in to a “full post” in due course but first notes from Krippendorf around coding noted that:

P127: “Recording takes place when observers, readers or analysts interpret what they see, read, or find and then state their experiences in the formal terms of an analysis, coding is the term content analysts use when this process is carried out according to observer-independent rules”

I find this interesting because… the “formal terms of an analysis” are emphasised in originating Grounded Theory (GT) T and hermeneutics approaches as key but often seem to be much diminished in contemporary practices of those “using GT” or other approaches to analysis influenced by GT. The formality of defining codes and consistently applying them is however very much inductive and open to continuous, data-driven revision.

However, it is the notion of observer independence where arguably the approach of content analysis differs so much from the inductive and interpretivist ideas framing much of qualitative analysis and the assumptions that proceed from that into suggestions of what software can do to assist such analysis. However, in CAQDAS packages “coding” can support or encompass both approaches – and I wonder to what extent this is a key source of the tensions, mistrust or the (frequent) misrepresentation of what CAQDAS packages “do” to analysis.

To be continued…

 

 

KWIC interfaces and concordances

This image from the excellent QD in Practice event organised at Leeds University really drove home to me just how powerful and useful KWIC (Key Words In Context) concordance displays can be.

kwicinarabic

In the image above I cannot even read the script – I don’t read arabic. Not only can I not read the script it is written from right-to-left, yet KWIC works.

I can see, without being able to understand, that there is a difference between lines 1, 2 3, lines 4 though 11 are the same, line 12 is different and lines 13 through 20 are the same in terms of the words in red that appear before (it’s R>L text, remember!) the highlighted keyword.

Since I first encountered KWIC in a module on corpus approaches to language teaching I have recognised that it has an incredible simplicity and power compared to many other ways of showing highlighted text.

From text to context – displaying search results in NVivo at Present

Compare it to this:

nvivowordsincontextsearchview1

Which is the results output from a text search in NVivo.

This is not a bad output, I see context in a similar was a KWIC concordance and can access the underlying data immediately. However, the appearance precludes some rather more important options KWIC enables.

Another way to reach this sort of word search is by running a word frequency query in NVivo – which will then create a list of words along with information on their length, their count, a weighted percentage (need to learn more on that) and a list of “similar words”.

The similar words are derived by including stemmed words – a process which has some issues associated with it which I’ll go into a little later. Here I’m going to focus on the representation of that information:

nvivowordfrequencyresults

So double-clicking on a word takes me to the same display as previously for a stemmed text search:

nvivowordsincontextsearchview2

Again not bad – I get some context and information on the source. And from it I can go and find the word in context in the original text by clicking the link – and the word is helpfully highlighted:

highlightedwordsincontext

A closer view – word trees

EDIT/UPDATE – from chatting with Silvana (and revisiting Kathleen’s comments in the NVivo Users Group). Word tree is indeed *very* similar to KWIC:

wordTree-NVivo

they show the key word in the middle and the branching before and after. The differences however are still important – while you can select the text to see connections:

wordTree-highlighted

What you cannot see as easily are the sentences across, or any variation. It’s a powerful tool that does much of the work of KWIC – but I’m not sure if the simplification comes at a cost. This is one for me to look at further – thanks to Kathleen for flagging it to me to cogitate on and explore further!

Of course MaxQDA does have KWIC 

What you can’t do or see easily with this… but could with KWIC

However, there are a bunch of things I can’t do or easily see which KWIC would enable:

  • Which words come before or after? (visible in word tree)
    • Consider for example the potentially very important differences between the pronouns that precede or follow a key term that is emerging as a theme or word – for example work/working or team/s and if or how these might very between groups or align with attributes you;re interested in (e.g. managers vs subordinates)
    • Consider for example the important differences between how use and used can appear as a verb, a modal auxiliary :
      • I used the software four years ago (verb, p/t)
      • I used to hate the software (quasi-modal)
      • I got used to the software (adjective phrase)
    • Which stems are associated? (Not sure if this is visible with word tree???)
      • Consider the spurious stemming that can occur e.g.
        • Office
        • Officer
        • Official
      • Which words are associated with particular stems or synonyms
        • Consider the difference between stems of
          • be, been, being
        • Compared to lemmatisation as
          • am, was, are, were

And here’s where the power yet simplicity of KWIC really holds potential for working with this sort of query and any coding from that. Consider what you can see when the data is presented in a KWIC concordance:

Ref 1:  0.01%

 a little while since I’ve

 use

d  Adobe Connect. Okay [pause] oh
Ref 2:  0.02%

 STS and how you’ve been

 using

 caqdas software, but it’s just
Ref 3:  0.02%

 that particularly made it seem

 use

ful or relevant or drew you
Ref 4:  0.02%

 ANT, but nevertheless he is

 using

 some of the principles of
Ref 5:  0.01%

 by Actor-network theory have

 use

d  software in their research. Erm
Ref 6:  0.02%

 poll is people who are

 using

 CAQDAS packages, some is people
Ref 7:  0.02%

 is people who are not

 using

 those. Erm, and some is
Ref 8:  0.02%

 some is people who are

 using

 a mixture of-, a sort
Ref 9:  0.02%

 wondered, what software are you

 using

? Erm, and one info [skip
Ref 10:  0.02%

 you know, beca

 use

-,  I start using what I knew at that
Ref 11:  0.02%

 start my PhD, we start

 using

 a specific software that I
Ref 12:  0.02%

 software that I had been

 using

 before, which is a qualitative
Ref 13:  0.01%

 study, then I have to

 use

  something that I knew and
Ref 14:  0.01%

 with Atlas T, and I

 use

  it-, I will explain it
Ref 15:  0.01%

 but later …[15.34] Then I

 use

d  Atlas T from the very
Ref 16:  0.01%

 the very beginning, and I

 use

d  it only to qualify all
Ref 17:  0.01%

 of my research. Erm, the

 use

 of Atlas T was useful
Ref 18:  0.02%

my

 use

of Atlas T was useful at some extent,
Ref 19:  0.01%

 best tool that I can

 use

, but I will explain it
Ref 20:  0.01%

 apply principles of ANT and

 use

  a specific software?’ [18.54] So
Ref 21:  0.01%

 of mine, err, quite frequently

 use

s the phrase ‘auto-magical’, and
Ref 22:  0.02%

 understand how ANTA can be

 use

ful in that sense. Of course
Ref 23:  0.02%

 learning, analytics, big data and

 using

 those special softwares, but I
Ref 24:  0.01%

 didn’t get how I can

 use

  it for my research, really
Ref 25:  0.01%

 and show me how you

 use

  Atlas.ti that would be really
Ref 26:  0.01%

 tools and options you do

 use

, that have supported you the
Ref 27:  0.02%

 broken.’ So which-, so you’re

 using

 Atlas T on a Mac
Ref 28:  0.01%

 Yes I [skip]-, I’m just

 use

[skip] [25.47] Steve W Okay
Ref 29:  0.02%

 finished my thesis, I am

 using

 [skip] as a module from
Ref 30:  0.02%

 you. This paper is about

 using

 ANT principles through my research
Ref 31:  0.01%

 yesterday found that I can

 use

  AtlasT not in my Windows
Ref 33:  0.02%

 with statements from other documents

 using

 categories of analysis. I mean
Ref 34:  0.01%

 you generate and did you

 use

? Alberto There is no [unclear

The power and importance of sorting

What I would like to be able to see is the kind of output shown above as an option along with the normal contextual view. I would want to be able to sort it by the middle column and/or the words immediately preceding or following that. This then really helps spot patterns:

Loc %

Text 1

Stem

Text 2
Ref 13:  0.01%

 study, then I have to

 use

  something that I knew and
Ref 14:  0.01%

 with Atlas T, and I

 use

  it-, I will explain it
Ref 17:  0.01%

 of my research. Erm, the

 use

 of Atlas T was useful
Ref 18:  0.02%

my

 use

of Atlas T was use ful at some extent, to some
Ref 19:  0.01%

 best tool that I can

 use

, but I will explain it
Ref 20:  0.01%

 apply principles of ANT and

 use

  a specific software?’ [18.54] So
Ref 24:  0.01%

 didn’t get how I can

 use

  it for my research, really
Ref 25:  0.01%

 and show me how you

 use

  Atlas.ti that would be really
Ref 26:  0.01%

 tools and options you do

 use

, that have supported you the
Ref 28:  0.01%

 Yes I [skip]-, I’m just

 use

[skip] [25.47] Steve W Okay
Ref 31:  0.01%

 yesterday found that I can

 use

  AtlasT not in my Windows
Ref 34:  0.01%

 you generate and did you

 use

? Alberto There is no [unclear
Ref 10:  0.02%

 you know, beca

 use

-,  I start using what I knew at that
Ref 1:  0.01%

 a little while since I’ve

 use

d  Adobe Connect. Okay [pause] oh
Ref 5:  0.01%

 by Actor-network theory have

 use

d  software in their research. Erm
Ref 15:  0.01%

 but later …[15.34] Then I

 use

d  Atlas T from the very
Ref 16:  0.01%

 the very beginning, and I

 use

d  it only to qualify all
Ref 3:  0.02%

 that particularly made it seem

 use

ful or relevant or drew you
Ref 22:  0.02%

 understand how ANTA can be

 use

ful in that sense. Of course
Ref 21:  0.01%

 of mine, err, quite frequently

 use

s the phrase ‘auto-magical’, and
Ref 2:  0.02%

 STS and how you’ve been

 using

 caqdas software, but it’s just
Ref 4:  0.02%

 ANT, but nevertheless he is

 using

 some of the principles of
Ref 6:  0.02%

 poll is people who are

 using

 CAQDAS packages, some is people
Ref 7:  0.02%

 is people who are not

 using

 those. Erm, and some is
Ref 8:  0.02%

 some is people who are

 using

 a mixture of-, a sort
Ref 9:  0.02%

 wondered, what software are you

 using

? Erm, and one info [skip
Ref 11:  0.02%

 start my PhD, we start

 using

 a specific software that I
Ref 12:  0.02%

 software that I had been

 using

 before, which is a qualitative
Ref 23:  0.02%

 learning, analytics, big data and

 using

 those special softwares, but I
Ref 27:  0.02%

 broken.’ So which-, so you’re

 using

 Atlas T on a Mac
Ref 29:  0.02%

 finished my thesis, I am

 using

 [skip] as a module from
Ref 30:  0.02%

 you. This paper is about

 using

 ANT principles through my research
Ref 33:  0.02%

 with statements from other documents

 using

 categories of analysis. I mean

This would help with viewing the associations created from a query.

The next level – making this KWIC view a way of shaping the associations of stems and synonyms

However, to really have power you would need to be able to use it to interact with and change those associations.  the functions I would really like (via right click or similar) are:

1 – remove link of stem (e.g. De-link office and officer as being the same word)

2 – remove synonym association (e.g.

3 – (Ideally – probably harder!)  create a link for lemmatisation and ideally save it to a dictionary or thesaurus. AND / OR differentiate on set of used to from another set of used to.

All of these are hugely facilitated by a KWIC concordance view – and hopefully some of this is fairly simple whilst other aspects may need to be on a longer list but I believe are really worthy of consideration especially for approaches oriented more towards content analysis and data mining rather than inductive analysis.

Praxis Blog: Experiential and Reflexive Ideas for Enhancing Computer Assistance for QDA

Introducing this blog – an outlet for my emerging ideas and experiences in considering As outlined elsewhere I am undertaking a research project exploring the influences on software choice and the way that STS researchers theorise, understand and use software in their research.

Part of this research project involves comparative analysis of the data set in differnet packages (to date this has primarily been NVivo 11 Pro , ATLAS.ti 7 and ATLAS.ti Mac) which has led to some pretty fundamental challenges around synchronising transcripts with media across these different packages.

More recently this is evolving to (finally) include ATLAS.ti 8. I have made some initial engagements in using Leximancer

I am hoping, due to some synergies and cross-overs with a “Big Data” project I am involved in to systematically analyse student survey data at Lancaster University to also bring in analysis in NVivo 11 Plus as well as QDA Miner/WordSTAT.

I am drawing on these experiences here to write about emerging ideas, thoughts and suggestions for using features, achieving analytic aims in different ways in different packages and features and enhancements I would like to see.