Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

You're right; I'm sorry I missed discussing that. Our testing methodology is to run each test at least 2 times and if the second test is within 5 percent of the first, to average the two results. If there's a greater than 5 percent difference, we run multiple tests until we can determine which result was abnormal and why.

It's not THE most scientific methodology, but it's a decent balance between time and accuracy. In the case of our most recent round of tests, none of the second results were more than 4 percent off the first test.



I'm really pleased to hear that. Given the lack of discussion I had assumed the worst, but I'm glad to be wrong, and sorry to presume otherwise. Thanks a lot for following up.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: