How to best illustrate trends/distributions of many data points?

[Peter Sommer]

Hi everyone,

I have a lot of data from an experiment and would like to find the best way to display it - been trying it for hours and about to give up.
I want to display two data points per individual: initial behavior and difference in behavior (= end behavior - initial behavior). I have data from two different treatments and regressions suggest that there are clear differences, but I find it hard to illustrate it. I have tried everything from area plots to scatter plots to variations of heat maps or simple fitting lines, but nothing looks clear. Here are two unsuccessful examples (x-axis: initial behavior, y-axis: change):



It's really not clear from any of these two pictures what's going on - I would greatly appreciate suggestions for how to display it more clearly.
Thank you!

[Maarten Buis]

If you give us an abstract of the data you want to visualize then we can experiment for you. Right now I would have to make way too many assumptions about what you mean, and will probably get some of those wrong, which would result in an answer that is useless to you. To give an extract type in Stata help dataex

[Marcos Almeida]

A regression analysis followed by - margin- and - marginsplot - commands should do the trick.

[Peter Sommer]

If you give us an abstract of the data you want to visualize then we can experiment for you. Right now I would have to make way too many assumptions about what you mean, and will probably get some of those wrong, which would result in an answer that is useless to you. To give an extract type in Stata help dataex

It's a lot of data and dataex only provides a partial output. It also doesn't really show anything that I haven't explained already. Two observations per participant, many observations in total. I tried a heatmap because many of the 'change' data points occur more frequently than other, also tried to illustrate it with 'bubbles' that would indicate the size. Everything looked disappointing so far.

* Example generated by -dataex-. To install: ssc install dataex
clear
input float change byte Treatment int SubjectNo byte Period
 100 1   1  1
 100 1   1 20
  40 1   2  1
  50 1   2 20
  50 1   3  1
  50 1   3 20
   0 1   4  1
-100 1   4 20
-100 1   5  1
-100 1   5 20
 -50 1   6  1
-100 1   6 20
 -60 1   7  1
 -60 1   7 20
 -30 1   8  1
 -30 1   8 20
   0 1   9  1
   0 1   9 20
  40 1  10  1
   0 1  10 20
  50 1  11  1
  50 1  11 20
   0 1  12  1
   0 1  12 20
-100 1  13  1
-100 1  13 20
-100 1  14  1
-100 1  14 20
 100 1  15  1
 100 1  15 20
 -10 1  16  1
 -50 1  16 20
 -69 1  65  1
-100 1  65 20
 -50 1  66  1
 -50 1  66 20
 -60 1  67  1
-100 1  67 20
-100 1  68  1
-100 1  68 20
-100 1  69  1
-100 1  69 20
 -50 1  70  1
 -60 1  70 20
   0 1  71  1
 -50 1  71 20
  40 1  72  1
 -30 1  72 20
 -50 1  73  1
 -60 1  73 20
  50 1  74  1
 100 1  74 20
-100 1  75  1
-100 1  75 20
   0 1  76  1
   0 1  76 20
   0 1  77  1
 -50 1  77 20
  50 1  78  1
   0 1  78 20
  60 1  79  1
   0 1  79 20
 -50 1  80  1
 -50 1  80 20
   0 1  97  1
  40 1  97 20
-100 1  98  1
-100 1  98 20
 100 1  99  1
 100 1  99 20
-100 1 100  1
-100 1 100 20
 -50 1 101  1
 -50 1 101 20
  50 1 102  1
 100 1 102 20
-100 1 103  1
-100 1 103 20
   0 1 104  1
  50 1 104 20
  50 1 105  1
  50 1 105 20
   0 1 106  1
 -50 1 106 20
  80 1 107  1
 100 1 107 20
  50 1 108  1
  50 1 108 20
-100 1 109  1
 -60 1 109 20
   0 1 110  1
   0 1 110 20
 -50 1 111  1
-100 1 111 20
   0 1 112  1
-100 1 112 20
   0 1 169  1
   0 1 169 20
 -50 1 170  1
 -10 1 170 20
end

------------------ copy up to and including the previous line ------------------

Listed 100 out of 1684 observations
Use the count() option to list more

[Peter Sommer]

A regression analysis followed by - margin- and - marginsplot - commands should do the trick.

I'm not sure this really helps my case. I'm not trying to illustrate any effects, but the relationship between two data points on the individual level in the form of a nice graphic.

[Maarten Buis]

You seem surprised that we want an extract from your data. Remember, we know nothing about your project. All we have is the text of your question. As self-evident as you believe that text is, it can be interpreted in many ways. The problem with doing an analysis or programming or creating a graph is that those details matter a lot. For example, Marcos interpreted your question in a different way than you intended, and you got an answer that did not help you. On a more practical level, the most useful answer you could get is some example code that you can adept. But example code needs data. It would be much easier for you to adept our example to your problem if the example data is already your data. We are not trying to be difficult, we are just trying to figure out what the question is you want answered and give you an answer that is most useful to you.

Having said all that, there is still a problem with the data you gave us: You want something like plotting the initial condition and the change. For that you need both the initial condition and the change. What you gave us was only the change. Can you give us both?