Something that has not been an issue in RF literature are the system requirements and performance of the systems. The techniques described in this paper all have a theoretical lower time bound that is linear in the size of the judged document set and number of terms indexed in those documents. It is entirely unclear how much time an implementation of any of these techniques takes. There are also questions regarding space. In an IR system that does not use Relevance Feedback, once a document has been inserted into its index, the IR system has no further need of it, and thus space can be saved by discarding the document. On the other hand, RF techniques require the document to be present, and thus additional space is necessary to store the document. In addition, there will be additional I/O overhead as the document is accessed.
Most of the popular Web search systems currently have document collections of between 30 and 55 million documents [10], receive about 5 million queries per day, and are able to satisfy each query within a few seconds. In order to accomplish this, these systems are running on top-of-the-line servers with multiple CPUs and gigabytes of memory. It is an open question as to the amount of additional resource requirements a Relevance Feedback system would consume, as well as if any such system could satisfy queries quickly enough to meet user expectations.