searching module

This module contains classes and functions related to searching the index.

Searcher class

class whoosh.searching.Searcher(ixreader, weighting=<class 'whoosh.scoring.BM25F'>)

Wraps an IndexReader object and provides methods for searching the index.

Parameters:
document(**kw)

Convenience method returns the stored fields of a document matching the given keyword arguments, where the keyword keys are field names and the values are terms that must appear in the field.

This method is equivalent to:

searcher.stored_fields(searcher.document_number(<keyword args>))

Where Searcher.documents() returns a generator, this function returns either a dictionary or None. Use it when you assume the given keyword arguments either match zero or one documents (i.e. at least one of the fields is a unique key).

>>> stored_fields = searcher.document(path=u"/a/b")
>>> if stored_fields:
...   print stored_fields['title']
... else:
...   print "There is no document with the path /a/b"
document_number(**kw)

Returns the document number of the document matching the given keyword arguments, where the keyword keys are field names and the values are terms that must appear in the field.

>>> docnum = searcher.document_number(path=u"/a/b")

Where Searcher.document_numbers() returns a generator, this function returns either an int or None. Use it when you assume the given keyword arguments either match zero or one documents (i.e. at least one of the fields is a unique key).

Return type:int
document_numbers(**kw)

Returns a generator of the document numbers for documents matching the given keyword arguments, where the keyword keys are field names and the values are terms that must appear in the field.

>>> docnums = list(searcher.document_numbers(emailto=u"matt@whoosh.ca"))
documents(**kw)

Convenience method returns the stored fields of a document matching the given keyword arguments, where the keyword keys are field names and the values are terms that must appear in the field.

Returns a generator of dictionaries containing the stored fields of any documents matching the keyword arguments.

>>> for stored_fields in searcher.documents(emailto=u"matt@whoosh.ca"):
...   print "Email subject:", stored_fields['subject']
field(fieldid)
Returns the whoosh.fields.Field object for the given field name.
fieldname_to_num(fieldid)
Returns the field number of the given field name.
fieldnum_to_name(fieldnum)
Returns the field name corresponding to the given field number.
idf(fieldid, text)
Calculates the Inverse Document Frequency of the current term. Subclasses may want to override this.
key_terms(docnums, fieldname, numterms=5, model=<class 'whoosh.classify.Bo1Model'>, normalize=True)

Returns the ‘numterms’ most important terms from the documents listed (by number) in ‘docnums’. You can get document numbers for the documents your interested in with the document_number() and document_numbers() methods.

>>> docnum = searcher.document_number(path=u"/a/b")
>>> keywords = list(searcher.key_terms([docnum], "content"))

“Most important” is generally defined as terms that occur frequently in the top hits but relatively infrequently in the collection as a whole.

Parameters:
  • fieldname – Look at the terms in this field. This field must store vectors.
  • docnums – A sequence of document numbers specifying which documents to extract key terms from.
  • numterms – Return this number of important terms.
  • model – The classify.ExpansionModel to use. See the classify module.
reader()
Returns the underlying IndexReader.
search(query, limit=5000, sortedby=None, reverse=False, minscore=0.0001)

Runs the query represented by the query object and returns a Results object.

Parameters:
  • query – a whoosh.query.Query object.
  • limit – the maximum number of documents to score. If you’re only interested in the top N documents, you can set limit=N to limit the scoring for a faster search.
  • sortedby

    if this parameter is not None, the results are sorted instead of scored. If this value is a string, the results are sorted by the field named in the string. If this value is a list or tuple, it is assumed to be a sequence of strings and the results are sorted by the fieldnames in the sequence. Otherwise ‘sortedby’ should be a scoring.Sorter object.

    The fields you want to sort by must be indexed.

    For example, to sort the results by the ‘path’ field:

    searcher.find(q, sortedby = "path")
    

    To sort the results by the ‘path’ field and then the ‘category’ field:

    searcher.find(q, sortedby = ("path", "category"))
    

    To use a sorting object:

    searcher.find(q, sortedby = scoring.FieldSorter("path", key=mykeyfn))
    

    Using a string or tuple simply instantiates a whoosh.scoring.FieldSorter or whoosh.scoring.MultiFieldSorter object for you. To get a custom sort order, instantiate your own FieldSorter with a key argument, or write a custom whoosh.scoring.Sorter class.

    FieldSorter and MultiFieldSorter cache the document order, using 4 bytes times the number of documents in the index, and taking time to cache. To increase performance, instantiate your own sorter and re-use it (but remember you need to recreate it if the index changes).

  • reverse – if sortedby is not None, this reverses the direction of the sort.
  • minscore – the minimum score to include in the results.
Return type:

Results

Results class

class whoosh.searching.Results(searcher, query, scored_list, docvector, scores=None, runtime=0)

This object is returned by a Searcher. This object represents the results of a search query. You can mostly use it as if it was a list of dictionaries, where each dictionary is the stored fields of the document at that position in the results.

Parameters:
  • searcher – the Searcher object that produced these results.
  • query – the original query that created these results.
  • scored_list – an ordered list of document numbers representing the ‘hits’.
  • docvector – a BitVector object where the indices are document numbers and an ‘on’ bit means that document is present in the results.
  • scores – a list of scores corresponding to the document numbers in scored_list, or None if no scores are available.
  • runtime – the time it took to run this search.
copy()
Returns a copy of this results object.
docnum(n)
Returns the document number of the result at position n in the list of ranked documents. Use __getitem__ (i.e. Results[n]) to get the stored fields directly.
extend(results)

Appends hits from ‘results’ (that are not already in this results object) to the end of these results.

Parameter:results – another results object.
filter(results)
Removes any hits that are not also in the other results object.
key_terms(fieldname, docs=10, numterms=5, model=<class 'whoosh.classify.Bo1Model'>, normalize=True)

Returns the ‘numterms’ most important terms from the top ‘numdocs’ documents in these results. “Most important” is generally defined as terms that occur frequently in the top hits but relatively infrequently in the collection as a whole.

Parameters:
  • fieldname – Look at the terms in this field. This field must store vectors.
  • docs – Look at this many of the top documents of the results.
  • terms – Return this number of important terms.
  • model – The classify.ExpansionModel to use. See the classify module.
Returns:

list of unicode strings.

score(n)
Returns the score for the document at the Nth position in the list of results. If the search was not scored, returns None.
scored_length()
Returns the number of RANKED documents. Note this may be fewer than the total number of documents the query matched, if you used the ‘limit’ keyword of the Searcher.search() method to limit the scoring.
upgrade(results, reverse=False)

Re-sorts the results so any hits that are also in ‘results’ appear before hits not in ‘results’, otherwise keeping their current relative positions. This does not add the documents in the other results object to this one.

Parameters:
  • results – another results object.
  • reverse – if True, lower the position of hits in the other results object instead of raising them.
upgrade_and_extend(results)

Combines the effects of extend() and increase(): hits that are also in ‘results’ are raised. Then any hits from ‘results’ that are not in this results object are appended to the end of these results.

Parameter:results – another results object.
class whoosh.searching.ResultsPage(results, pagenum, pagelen=10)

Represents a single page out of a longer list of results, as returned by whoosh.searching.Searcher.search_page(). Supports a subset of the interface of the Results object, namely getting stored fields with __getitem__ (square brackets), iterating, and the score() and docnum() methods.

The offset attribute contains the results number this page starts at (numbered from 0). For example, if the page length is 10, the offset attribute on the second page will be 10.

The pagecount attribute contains the number of pages available.

The pagenum attribute contains the page number. This may be less than the page you requested if the results had too few pages. For example, if you do:

ResultsPage(results, 5)

but the results object only contains 3 pages worth of hits, pagenum will be 3.

The pagelen attribute contains the number of results on this page (which may be less than the page length you requested if this is the last page of the results).

The total attribute contains the total number of hits in the results.

>>> mysearcher = myindex.searcher()
>>> pagenum = 2
>>> page = mysearcher.find_page(pagenum, myquery)
>>> print("Page %s of %s, results %s to %s of %s" %
...       (pagenum, page.pagecount, page.offset+1, page.offset+page.pagelen, page.total))
>>> for i, fields in enumerate(page):
...   print("%s. %r" % (page.offset + i + 1, fields))
>>> mysearcher.close()
Parameters:
  • results – a Results object.
  • pagenum – which page of the results to use, numbered from 1.
  • pagelen – the number of hits per page.
docnum(n)
Returns the document number of the hit at the nth position on this page.
score(n)
Returns the score of the hit at the nth position on this page.

Table Of Contents

Previous topic

scoring module

Next topic

spelling module

This Page