Class: Evaluate

Parallel program evaluator (dspy.Evaluate).

 classDiagram
    class Evaluate
    click Evaluate href "../Evaluate/"
      NamedThing <|-- Evaluate
        click NamedThing href "../NamedThing/"

      Evaluate : description

      Evaluate : devset





        Evaluate --> "*" Example : devset
        click Example href "../Example/"



      Evaluate : display_progress

      Evaluate : display_table

      Evaluate : failure_score

      Evaluate : id

      Evaluate : max_errors

      Evaluate : metric

      Evaluate : name

      Evaluate : num_threads

      Evaluate : provide_traceback

      Evaluate : save_as_csv

      Evaluate : save_as_json

Inheritance

NamedThing
- Evaluate

Slots

Name	Cardinality and Range	Description	Inheritance
devset	* Example	Dev / evaluation split	direct
metric	0..1 String	Dotted import path of the metric callable (or its name)	direct
num_threads	0..1 Integer		direct
display_progress	0..1 Boolean		direct
display_table	0..1 Integer	0 to hide, N>0 to show the first N rows	direct
max_errors	0..1 Integer		direct
provide_traceback	0..1 Boolean		direct
failure_score	0..1 Float		direct
save_as_csv	0..1 String		direct
save_as_json	0..1 String		direct
id	1 Identifier	Stable opaque identifier for an instance	NamedThing
name	0..1 String	Short human-readable name	NamedThing
description	0..1 String	Free-form description	NamedThing

In Subsets

Evaluation

Identifier and Mapping Information

Annotations

property	value
dspy_source_paths	['dspy/evaluate/evaluate.py']
dspy_source_urls	['https://github.com/stanfordnlp/dspy/blob/main/dspy/evaluate/evaluate.py']

Schema Source

from schema: https://w3id.org/lmodel/dspy

Mappings

Mapping Type	Mapped Value
self	dspy:Evaluate
native	dspy:Evaluate
related	iso22989:TrustworthinessPropertyType, iso42001:AuditFindingType, mellea:RequirementSpec, nist_ai_100_1:Function
close	ai:Benchmarking, nexus:AiEval

LinkML Source

Direct

name: Evaluate
annotations:
  dspy_source_paths:
    tag: dspy_source_paths
    value:
    - dspy/evaluate/evaluate.py
  dspy_source_urls:
    tag: dspy_source_urls
    value:
    - https://github.com/stanfordnlp/dspy/blob/main/dspy/evaluate/evaluate.py
description: Parallel program evaluator (``dspy.Evaluate``).
in_subset:
- evaluation
from_schema: https://w3id.org/lmodel/dspy
close_mappings:
- ai:Benchmarking
- nexus:AiEval
related_mappings:
- iso22989:TrustworthinessPropertyType
- iso42001:AuditFindingType
- mellea:RequirementSpec
- nist_ai_100_1:Function
is_a: NamedThing
slots:
- devset
- metric
- num_threads
- display_progress
- display_table
- max_errors
- provide_traceback
- failure_score
- save_as_csv
- save_as_json

Induced

name: Evaluate
annotations:
  dspy_source_paths:
    tag: dspy_source_paths
    value:
    - dspy/evaluate/evaluate.py
  dspy_source_urls:
    tag: dspy_source_urls
    value:
    - https://github.com/stanfordnlp/dspy/blob/main/dspy/evaluate/evaluate.py
description: Parallel program evaluator (``dspy.Evaluate``).
in_subset:
- evaluation
from_schema: https://w3id.org/lmodel/dspy
close_mappings:
- ai:Benchmarking
- nexus:AiEval
related_mappings:
- iso22989:TrustworthinessPropertyType
- iso42001:AuditFindingType
- mellea:RequirementSpec
- nist_ai_100_1:Function
is_a: NamedThing
attributes:
  devset:
    name: devset
    description: Dev / evaluation split.
    from_schema: https://w3id.org/lmodel/dspy
    rank: 1000
    owner: Evaluate
    domain_of:
    - Evaluate
    - Dataset
    range: Example
    multivalued: true
    inlined: true
    inlined_as_list: true
  metric:
    name: metric
    description: Dotted import path of the metric callable (or its name).
    from_schema: https://w3id.org/lmodel/dspy
    rank: 1000
    owner: Evaluate
    domain_of:
    - Teleprompter
    - Evaluate
    range: string
  num_threads:
    name: num_threads
    from_schema: https://w3id.org/lmodel/dspy
    rank: 1000
    owner: Evaluate
    domain_of:
    - Parallel
    - BootstrapFewShotWithRandomSearch
    - GEPA
    - BootstrapFinetune
    - Evaluate
    - Settings
    range: integer
    minimum_value: 1
  display_progress:
    name: display_progress
    from_schema: https://w3id.org/lmodel/dspy
    rank: 1000
    ifabsent: boolean(False)
    owner: Evaluate
    domain_of:
    - Evaluate
    range: boolean
  display_table:
    name: display_table
    description: 0 to hide, N>0 to show the first N rows.
    from_schema: https://w3id.org/lmodel/dspy
    rank: 1000
    owner: Evaluate
    domain_of:
    - Evaluate
    range: integer
  max_errors:
    name: max_errors
    from_schema: https://w3id.org/lmodel/dspy
    rank: 1000
    owner: Evaluate
    domain_of:
    - Parallel
    - BootstrapFewShot
    - Evaluate
    - Settings
    range: integer
    minimum_value: 0
  provide_traceback:
    name: provide_traceback
    from_schema: https://w3id.org/lmodel/dspy
    rank: 1000
    ifabsent: boolean(False)
    owner: Evaluate
    domain_of:
    - Parallel
    - Evaluate
    - Settings
    range: boolean
  failure_score:
    name: failure_score
    from_schema: https://w3id.org/lmodel/dspy
    rank: 1000
    ifabsent: float(0.0)
    owner: Evaluate
    domain_of:
    - Evaluate
    range: float
  save_as_csv:
    name: save_as_csv
    from_schema: https://w3id.org/lmodel/dspy
    rank: 1000
    owner: Evaluate
    domain_of:
    - Evaluate
    range: string
  save_as_json:
    name: save_as_json
    from_schema: https://w3id.org/lmodel/dspy
    rank: 1000
    owner: Evaluate
    domain_of:
    - Evaluate
    range: string
  id:
    name: id
    description: Stable opaque identifier for an instance.
    from_schema: https://w3id.org/lmodel/dspy
    rank: 1000
    slot_uri: schema:identifier
    identifier: true
    owner: Evaluate
    domain_of:
    - NamedThing
    range: Identifier
    required: true
  name:
    name: name
    description: Short human-readable name.
    from_schema: https://w3id.org/lmodel/dspy
    rank: 1000
    slot_uri: rdfs:label
    owner: Evaluate
    domain_of:
    - NamedThing
    - LMToolCallPart
    - LMMessage
    - LMToolSpec
    range: string
  description:
    name: description
    description: Free-form description.
    from_schema: https://w3id.org/lmodel/dspy
    rank: 1000
    slot_uri: dct:description
    owner: Evaluate
    domain_of:
    - NamedThing
    - LMToolSpec
    range: string