Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • #16
    Originally posted by daniel klein View Post
    So no, it's not just a cosmetic detail. It fundamentally affects the results.
    Certainly. I was not implying that the process stops with the four extreme values derived, merely that these can be easily used to get to the two absolute extremes.
    Thank you for your input,

    Jan

    Comment


    • #17
      Originally posted by Jan Kabatek View Post
      I was not implying that the process stops with the four extreme values derived, merely that these can be easily used to get to the two absolute extremes.
      I see where you're coming from. You have to sort once either way. Picking the two smalles and two largest raw values then avoids taking absolutes for the entire vector.

      Out of curiousity, which kind of data are you working with? If you're worried about the speed of sorting, we must be well beyond a (couple of) million observations. At the same time, you're concerend that the two most extreme values out of misslions account for omre than two thirds of the total sum.

      Comment


      • #18
        Originally posted by daniel klein View Post
        Out of curiousity, which kind of data are you working with? If you're worried about the speed of sorting, we must be well beyond a (couple of) million observations. At the same time, you're concerend that the two most extreme values out of misslions account for omre than two thirds of the total sum.
        Population-level registers. The numbers of observations tend to be in the ballpark of 1M-100M.

        That alone would not be excessive, however I am sharing the same hardware with many other users. As a result, the experience is similar to working on a standard office PC built about 15 years ago.

        The extreme-value criterion is indeed rather silly, considering the sample sizes we work with (AFAICT, it has been introduced to prevent re-identification of very rich individuals in very small subpopulations). But the rule is in place, and the agency is quite keen on it...

        Comment

        Working...
        X