Varying background lines in fabplot, by variable value [OR, loop in twoway]

[Elizabeth Pisani]

I am trying to generate a graph comparing time series runs between samples in different arms of an experiment. Each sample is defined by variable barcode, but one or more samples can be grouped in to an arm. I have one observation per timepoint per barcode. Example here (with apologies for length of excerpt:

* Example generated by -dataex-. For more info, type help dataex
clear
input str5 experiment str9 barcode float(seconds freq) byte arm
"01_02" "allo_014"  41.10912  9.626477 3
"01_02" "allo_014"  44.23299  9.607565 3
"01_02" "allo_014"  47.59423   9.56974 3
"01_02" "allo_014"   51.2109  9.531915 3
"01_02" "allo_014"   55.1024  9.513002 3
"01_02" "allo_014"  59.28961  9.475178 3
"01_02" "allo_014"  63.79501  9.437352 3
"01_02" "allo_014"  68.64277 9.4562645 3
"01_02" "allo_014"   73.8589  9.437352 3
"01_02" "allo_014"  79.47141   9.41844 3
"01_02" "allo_014"  85.51041   9.41844 3
"01_02" "allo_014"  92.00832  9.437352 3
"01_02" "allo_014"        99  9.399528 3
"01_02" "allo_014" 106.52297   9.41844 3
"01_02" "allo_014" 114.61761   9.41844 3
"01_02" "allo_014" 123.32736   9.41844 3
"01_02" "allo_014" 132.69896   9.41844 3
"01_02" "allo_014" 142.78271  9.399528 3
"01_02" "allo_014"  153.6327   9.41844 3
"01_02" "allo_014"  165.3072  9.399528 3
"01_02" "allo_014" 177.86884  9.361702 3
"01_02" "allo_014" 191.38503  9.437352 3
"01_02" "allo_014"  205.9283  9.437352 3
"01_02" "allo_014"  221.5767  9.399528 3
"01_02" "allo_014" 238.41425  9.380614 3
"01_02" "allo_014" 256.53125  9.399528 3
"01_02" "allo_014" 276.02496  9.399528 3
"01_02" "allo_014"       297  9.399528 3
"01_02" "allo_014"  319.5689 9.4562645 3
"01_02" "allo_014"  343.8528  9.437352 3
"01_02" "allo_027"  192.8848  9.021276 2
"01_02" "allo_027"  255.8488  9.078014 2
"01_02" "allo_027" 274.57108  9.059102 2
"01_02" "allo_027" 294.66336  9.078014 2
"01_02" "allo_027" 316.22595  9.078014 2
"01_02" "allo_233"  40.78759  8.113476 1
"01_02" "allo_233"   43.7723   8.11978 1
"01_02" "allo_233"  50.41295  8.179668 1
"01_02" "allo_233"  54.10202  8.037825 1
"01_02" "allo_233"  58.06104  7.943262 1
"01_02" "allo_233"  62.30978  8.132387 1
"01_02" "allo_233" 66.869415  7.895981 1
"01_02" "allo_233" 71.762726  7.955871 1
"01_02" "allo_233"  77.01411  7.918046 1
"01_02" "allo_233"  82.64977  8.132388 1
"01_02" "allo_233"  88.69783  7.867612 1
"01_02" "allo_233"  95.18847  7.810874 1
"01_02" "allo_233" 102.15408  7.747833 1
"01_02" "allo_233"  109.6294  7.546099 1
"01_02" "allo_233" 117.65176    7.6974 1
"01_02" "allo_233" 126.26117  7.385343 1
"01_02" "allo_233" 135.50058  7.596533 1
"01_02" "allo_233"  145.4161  7.602837 1
"01_02" "allo_233"  156.0572  7.609141 1
"01_02" "allo_233"   167.477  7.615445 1
"01_02" "allo_233"  179.7325  7.602837 1
"01_02" "allo_233"  192.8848  7.602837 1
"01_02" "allo_233"  206.9995  7.621749 1
"01_02" "allo_233" 222.14713  7.404256 1
"01_02" "allo_233"  238.4032  7.596533 1
"01_02" "allo_233"  255.8488  7.634358 1
"01_02" "allo_233" 274.57108  8.113476 1
"01_02" "allo_233" 294.66336    8.1513 1
"01_02" "allo_233" 316.22595 8.1229315 1
"01_02" "allo_233"  339.3664  7.735225 1
"01_02" "allo_234"  40.78759   9.34279 2
"01_02" "allo_234"   43.7723  9.191489 2
"01_02" "allo_234"  46.97543  9.115839 2
"01_02" "allo_234"  50.41295  9.115839 2
"01_02" "allo_234"  54.10202  9.115839 2
"01_02" "allo_234"  58.06104  9.059102 2
"01_02" "allo_234" 66.869415  9.096927 2
"01_02" "allo_234" 71.762726  9.040189 2
"01_02" "allo_234"  82.64977  9.040189 2
"01_02" "allo_234" 102.15408  8.983452 2
"01_02" "allo_234" 117.65176  8.983452 2
"01_02" "allo_234" 126.26117  9.002364 2
"01_02" "allo_234" 135.50058  9.021276 2
"01_02" "allo_234"  145.4161  9.021276 2
"01_02" "allo_234"  192.8848  9.021276 2
"01_02" "allo_234"  206.9995  9.021276 2
"01_02" "allo_234" 222.14713  9.002364 2
"01_02" "allo_234"  238.4032  8.955083 2
"01_02" "allo_234"  255.8488  9.021276 2
"01_02" "allo_234" 274.57108  9.040189 2
"01_02" "allo_234" 294.66336  9.059102 2
"01_02" "allo_234" 316.22595  9.096927 2
"01_02" "allo_234"  339.3664  9.078014 2
"01_02" "allo_235"  40.78759   9.49409 2
"01_02" "allo_235"   43.7723  9.475178 2
"01_02" "allo_235"  46.97543 9.4562645 2
"01_02" "allo_235"  50.41295  9.248226 2
"01_02" "allo_235"  54.10202  9.078014 2
"01_02" "allo_235"  58.06104  9.229315 2
"01_02" "allo_235"  62.30978  9.182034 2
"01_02" "allo_235" 66.869415  9.021276 2
"01_02" "allo_235"  82.64977   9.16312 2
"01_02" "allo_235"  95.18847  9.078014 2
"01_02" "allo_235" 117.65176  9.134751 2
"01_02" "allo_235" 274.57108  9.021276 2
"01_02" "allo_236"   43.7723  9.314421 2
"01_02" "allo_236"  46.97543  9.304964 2
"01_02" "allo_236"  50.41295  9.134751 2
"01_02" "allo_236"  54.10202  9.125296 2
"01_02" "allo_236"  58.06104  9.096927 2
"01_02" "allo_236" 66.869415  9.078014 2
"01_02" "allo_236"  255.8488  9.021276 2
end
label values arm arm
label def arm 1 "falsified", modify
label def arm 2 "geniuine batch numbers", modify
label def arm 3 "new sample", modify

I will be repeating similar but non-identical analyses on a fair number of experiments, so would like to automate as much as possible.

I have been using fabplot (which I like very much) with the select option to compare a single sample against the rest, as follows:

use $uccworking/longdata.dta, clear
keep if experiment == "01_02"
keep freq seconds arm experiment barcode
local experiment = experiment[1]
recode arm .=3
lab def arm 1 "falsified" 2 "geniuine batch numbers" 3 "new sample"
lab val arm arm

*FULL SCALE
gen front = barcode if arm ==1
gsort -front  // local = only pulls value from first observation, so have to make the observation of interest first in set
local front = front

sort seconds
fabplot line freq second, by(barcode) front(connect) frontopts(lw(medium) lc(red) msymbol(diamond) mcolor(red) msize(tiny)) select(barcode == "`front'") ///
ytitle("Frequency, in KHz per second", size(small)) ylabel(1(1)10, labsize(vsmall) angle(0)) yscale(r(0 10)) ///
xtitle("Time in seconds", size(small)) ///
xlab(50(50)1000, format(%4.0f) labsize(vsmall)) ///
title("Experiment `experiment': Branded allopurinol", size(small)) ///
plotregion(lstyle(none))
graph export "$uccresults/`experiment'_arm1_fullscale.png", replace

This is close to what I want. The "front" variable is always arm 1. Ideally, though, I would like to be able to vary the colours of the background lines by the other values of "arm"; never more than arms 2 and 3. In this example, there is 1 sample in arm 3 and there are 4 in arm 2.

Question 1: Is this possible in fabplot?
Question 2: if yes, is there a way of adding a legend?
Question 3: Is it possible to suppress the automatically included title band?

Possible alternatives:

Tsline is out for my purposes because the time variable must be an integer.
I have explored using xtline with addplot, but defining all the plotnopts seems more of a faff than going for a straight twoway. Which brings me to another question: Is it possible in some way to cycle through values of a variable as added plots within a twoway?

For the values of arm that only have one barcode, I can do this:

[CODE]
twoway scatter freq second if arm ==1, connect(direct) msize(vsmall) msymbol(triangle) mcolor(red) lcolor(red) || ///
scatter freq second if arm==3, connect(direct) msize(vsmall) msymbol(square) mcolor(blue) lcolor(blue) || ///
xtitle("Time in seconds", size(small)) ///
xlab(, format(%4.0f) labsize(vsmall)) ///
ytitle("Frequency, in KHz per second", size(vsmall)) ylabel(1(1)10, labsize(vsmall) angle(0)) yscale(r(0 10)) ///
title("Experiment `experiment': Branded allopurinol", size(small)) ///
legend(order(2 "Falsified" 1 "SM sample, genuine" 3 "New sample") size(vsmall) rows(1) region(lcolor(gray*.5))) ///
plotregion(lstyle(none))
[\CODE]

I could obviously add the four separate barcodes in arm = 2 manually:
[CODE]
|| scatter freq second if barcode == "allo_027", connect(direct) msize(vsmall) msymbol(circle) mcolor(green) lcolor(green) || ///
|| scatter freq second if barcode == "allo_234", connect(direct) msize(vsmall) msymbol(circle) mcolor(green) lcolor(green) || ///
|| scatter freq second if barcode == "allo_235", connect(direct) msize(vsmall) msymbol(circle) mcolor(green) lcolor(green) || ///
|| scatter freq second if barcode == "allo_236", connect(direct) msize(vsmall) msymbol(circle) mcolor(green) lcolor(green) ///
[\CODE]

However, they are all specified the same way (i.e. same colours and markers) and in some experiments there are up to 20 units in this arm. [Anticipating spagetti warning, we EXPECT these roughly to overlay one another on the graph, so there will be a block of arm 2 in green or I guess follwoing the fab convention gray, with arm 1 and arm 3 potentially diverging] Ideally, I'd like to run a loop or similar which automates the process for arms with more than 1 barcode. I have played around with some local options, but cannot get them to run within a twoway command overlaying plots on the same graph.

Thanks for any pointers you can give me.

[Nick Cox]

fabplot is from the Stata Journal. It doesn't support different colours for the series in back on purpose. and I don't think it will play with the new options such as colorvar().

In a strong sense the entire purpose is to make a simple contrast between each series in turn and the others, presented in subdued fashion.

That said, I've thought a little about generalising it, or varying it, to select groups of series at a time, but never got beyond thinking about a little.


Detail: The [\CODE] mark-up doesn't work; you need a forward slash there.

[Elizabeth Pisani]

Thanks, @Nick, and apologies for the inappropriate slash. If you have time to think a little more about that, I'm sure I would not be the only willing user.

Meanwhile, I will continue to try to find a solution involving a regular twoway command. Allow me to repost correctly the section of code I would like to be able to loop, within a two-way graph command which also includes other elements (as in my original post):

twoway scatter freq second if barcode == "allo_027", connect(direct) msize(vsmall) msymbol(circle) mcolor(green) lcolor(green) || ///
scatter freq second if barcode == "allo_234", connect(direct) msize(vsmall) msymbol(circle) mcolor(green) lcolor(green) || ///
scatter freq second if barcode == "allo_235", connect(direct) msize(vsmall) msymbol(circle) mcolor(green) lcolor(green) || ///
scatter freq second if barcode == "allo_236", connect(direct) msize(vsmall) msymbol(circle) mcolor(green) lcolor(green)

The barcodes (as well as the number of individual barcodes in each set) will differ in different graphs, so using local would be handy for that purpose also.

Thanks in advance for suggestions.

[Nick Cox]

I would use sepscatter here from SSC. https://www.statalist.org/forums/for...lable-from-ssc

gen barcode2 = substr(barcode, -3, 3)
sepscatter freq second if inlist(barcode2, "027", "234", "235", "236"), sep(barcode2) connect(l ..) legend(ring(0) col(1) pos(1)) xla(0(50)350)

sepscatter freq second if inlist(barcode2, "027", "234", "235", "236"), sep(barcode2) connect(l ..) mc(green ..) lc(green ..) msize(small ..) ms(O ..)legend(off) xla(0(50)350)