How We Know · Case 003 · encyclopedia

What GPT-4's 90th-percentile bar-exam claim compared

How did a historical simulated UBE score reported for GPT-4 become a roughly 90th-percentile claim, and how does the rank change when the comparison population changes?

Historical evidence through 2026-08-27 about one simulated UBE score and the comparison populations used to rank it. This evidence file does not describe current model behavior, practicing-lawyer quality, or general legal competence.

encyclopedia finding

historical simulated score documented; percentile is comparison-class dependent

What to do with that: Treat the historical score as a score, not as a universal percentile: name the exam administration and comparison population every time.

The documented result concerns one historical simulated-UBE run assembled from multiple score components.

Lineage accounting

The units count different things

1historical model runReport, paper, repository, and supplement share one experiment 3official administration sensitivitiesThe same score ranks differently across official Illinois charts 7rooted evidence unitsSeven units after editions and reused score inputs are collapsed 3unresolved boundariesVisible limits receive no invented value or broader interpretation

Shared capture: The report, research paper, repository, and supplements are manifestations of one historical model run; the Martinez analysis reuses that score and aggregate comparison data.

Warrant boundary: Percentile is not a property of the score alone. It depends on the administration, population, cutoff, interpolation, and model assumptions.

Sentence x-ray

What the selected record actually supports

Open a sentence to inspect its exact work, edition, span, retrieval, digest, and license chain.

01 OpenAI's launch-edition report displayed 298/400 and approximately 90th percentile for a simulated UBE.

Typed relation: support

Relation: relation-assertion-claim-launch-score-label em:dossier-evidence-relation:sha256:c4c25358513582c6f0bec9b1006974b13e32d524aff7d3f65ecd7107e0501b7f
Proposition: claim-launch-score-label em:dossier-proposition:sha256:c8893cb50e4e0702860b0e95d0bcff6b42ec4829435614188dfc775c3b790780
Assertion: assertion-claim-launch-score-label em:dossier-assertion:sha256:a3d71e28d0bf3602752a29727659782855a07b71d503841cfb82d25819cce382
Lineage: lineage-model-performance-root em:dossier-lineage:sha256:c8b81e09b0245e5c5302dd89be2d627f7658d7ab0d5de6d56adef3df0f65e145

Typed dependence

edge-author-social-collaborationlineage-model-performance-root → lineage-model-performance-rootem:dossier-evidence-relation:sha256:b616095462050178a4dedd71e5ad524effaa71d20808eceb206d203f9aae8204accepted_edge_id=edge-author-social-collaboration; accepted_dimension=author-social; from=lineage-model-performance-root; to=lineage-model-performance-root; effects=The vendor report and study are socially linked manifestations, not independent tests.
edge-citation-katz-to-martinezlineage-model-performance-root → lineage-martinez-analysis-rootem:dossier-evidence-relation:sha256:a8c16efd90634c0a152f0fdd7a0a02f8d844004ea05edec24ec4651bf00764ccaccepted_edge_id=edge-citation-katz-to-martinez; accepted_dimension=citation; from=lineage-model-performance-root; to=lineage-martinez-analysis-root; effects=Citation links the analysis lineage to the same reported score.
edge-data-reported-score-reuselineage-model-performance-root → lineage-martinez-analysis-rootem:dossier-evidence-relation:sha256:691d48bc190063d50b6e811c7455ec325b1f29bcc7452301694470e83211188aaccepted_edge_id=edge-data-reported-score-reuse; accepted_dimension=data; from=lineage-model-performance-root; to=lineage-martinez-analysis-root; effects=The re-analysis is not a new model-performance experiment. | Score validation changes warrant, not the participant or model-performance root count.
edge-material-shared-exam-itemslineage-model-performance-root → lineage-model-performance-rootem:dossier-evidence-relation:sha256:8f61eda50599e5c6c226c2dcb7818c8627f23979fd5b51669aa62e5d52f6d110accepted_edge_id=edge-material-shared-exam-items; accepted_dimension=material; from=lineage-model-performance-root; to=lineage-model-performance-root; effects=All score manifestations inherit the same exam-item materials.
edge-method-single-free-response-runlineage-model-performance-root → lineage-model-performance-rootem:dossier-evidence-relation:sha256:1ed5b88123954bc8da25cf9422c2df70e30d3569ddd99213e9cf5590bf8522c3accepted_edge_id=edge-method-single-free-response-run; accepted_dimension=method; from=lineage-model-performance-root; to=lineage-model-performance-root; effects=Repeated documents do not supply repeated-run independence.
edge-model-historical-snapshotslineage-model-performance-root → lineage-model-performance-rootem:dossier-evidence-relation:sha256:e7c0af1eebfe5fa400bcb3369655f40adf482a99cc91a370f6e6a586f847f133accepted_edge_id=edge-model-historical-snapshots; accepted_dimension=model; from=lineage-model-performance-root; to=lineage-model-performance-root; effects=The two snapshots are components of one reported experiment, not two evidence roots.
edge-score-component-compositelineage-model-performance-root → lineage-model-performance-rootem:dossier-evidence-relation:sha256:21278e42f97e114bc1442703ca96eea56725d7bb221a07e5c31f5adfc512b040accepted_edge_id=edge-score-component-composite; accepted_dimension=score; from=lineage-model-performance-root; to=lineage-model-performance-root; effects=297 and 298 are scoring variants within one experiment.

GPT-4 Technical Report

Table 1, PDF p. 5

Uniform Bar Exam (MBE+MEE+MPT) 298 / 400 (~90th)

Work: em:dossier-source-work:sha256:2c6c231a62e11f961467d990d3bd0ad2d4d4568fcd39891c16e6fe4622e9448b
Edition: em:dossier-edition:sha256:859fa96f390685d21c4db9633a6d9c7fec7abe76cdf8a1f8c0c9c82501b193eb
Edition digest: sha256:1703b719a1fad06284c9cde691496e2ee6cc9371ab7808d8abfc50ab3ed0c2f9
Span: em:dossier-span:sha256:959d593b53bd7991549571a95aa96133113958253fa6176851f1fb9bc6203534
Span digest: sha256:9b8313637a2bd3881a98d675793538fbeb77ac8ac64f945ca8ae2d962d3fd79a
Retrieval: No external retrieval record; repository audit span
License: arXiv non-exclusive distribution license; no Creative Commons license confirmed · arXiv non-exclusive distribution license; no Creative Commons license confirmed

GPT-4 Technical Report

Appendix A.1, PDF p. 23

The Uniform Bar Exam was run by our collaborators at CaseText and Stanford CodeX.

Work: em:dossier-source-work:sha256:2c6c231a62e11f961467d990d3bd0ad2d4d4568fcd39891c16e6fe4622e9448b
Edition: em:dossier-edition:sha256:859fa96f390685d21c4db9633a6d9c7fec7abe76cdf8a1f8c0c9c82501b193eb
Edition digest: sha256:1703b719a1fad06284c9cde691496e2ee6cc9371ab7808d8abfc50ab3ed0c2f9
Span: em:dossier-span:sha256:61f2be1da866e5764042a9022c4a0a0eb5d320a0df71f95a24a1a1eb5fd271ac
Span digest: sha256:e0b72dc61e4614a425e991b3fd9641db637b3516311659728f615bed0ea4100c
Retrieval: No external retrieval record; repository audit span
License: arXiv non-exclusive distribution license; no Creative Commons license confirmed · arXiv non-exclusive distribution license; no Creative Commons license confirmed

Appendix for GPT-4 passes the Bar Exam

footnote 5

While we are not fully convinced of the methodological approach taken in some subsequent analysis [78], we do agree that it would be better to consider the raw 297 UBE as falling within a range between 68th and 90th percentile (depending on the precise state and timing of the exam administration).

Work: em:dossier-source-work:sha256:880443d56eafdb224149d953f7de48e7550d53f814008551895b180ecb4b203b
Edition: em:dossier-edition:sha256:21621ede6a23bb3f98867c0918e5d9808d6dced5ca3f271a3a52e561810f2f7b
Edition digest: sha256:9d10cfedd531e62d96ea27bb78b00a9148e6b5d8135807836c5982e65a42416e
Span: em:dossier-span:sha256:477c595f89b6efc4f4fac04d627e65d95daa7929d10f76d3486a96c6c8bf8b06
Span digest: sha256:7e376175b36f8e2809bc74e378c7354858289a689370b70f82df2965d32b9026
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree · CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree

Appendix for GPT-4 passes the Bar Exam

§4(d), Table 7

{ "cells": [ { "cell_id": "cell-katz-gpt4-mbe", "column": "GPT-4", "row": "MBE", "text": "157 points" }, { "cell_id": "cell-katz-gpt4-mee", "column": "GPT-4", "row": "MEE", "text": "84 points" }, { "cell_id": "cell-katz-gpt4-mpt", "column": "GPT-4", "row": "MPT", "text": "56 points" }, { "cell_id": "cell-katz-gpt4-overall", "column": "GPT-4", "row": "overall score", "text": "297 points" } ], "format": "table-cell-transcription", "locator": "\u00a74(d), Table 7", "normalization": "exact-cell-text", "span_id": "span-katz-vor-components", "supports": "The version-of-record component scores sum to 297." }

Work: em:dossier-source-work:sha256:880443d56eafdb224149d953f7de48e7550d53f814008551895b180ecb4b203b
Edition: em:dossier-edition:sha256:21621ede6a23bb3f98867c0918e5d9808d6dced5ca3f271a3a52e561810f2f7b
Edition digest: sha256:9d10cfedd531e62d96ea27bb78b00a9148e6b5d8135807836c5982e65a42416e
Span: em:dossier-span:sha256:293687df5cf4b74d839237414173205591aacf4850103e1621aece4f21fb763b
Span digest: sha256:e6c808d4b4dcd243210518497a0c8788d06ee91d8571c474c13514c2c8afe04b
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree · CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree

percentile_analysis_new.Rmd

introduction, journal p. 584

The paper successfully replicates the MBE score of 158, but highlights several methodological issues in the grading of the MPT + MEE components of the exam, which call into question the validity of the essay score (140).

Work: em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b
Edition: em:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667
Edition digest: sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5
Span: em:dossier-span:sha256:47ce609849ae1ff1224df62ce9b453707dd9b7a93cd4413822c84178eac7f853
Span digest: sha256:11db34f6066f861f338266e98c11852743b727734f62c1a773ae257bf219e609
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed

Appendix for GPT-4 passes the Bar Exam

§3(a), materials

{ "format": "exact-segments", "locator": "\u00a73(a), materials", "normalization": "collapse-whitespace", "segments": [ { "segment_id": "segment-katz-mee-mpt-materials", "text": "For the MEE and the MPT, we collected the most recently released questions from the July 2022 Bar Examination." }, { "segment_id": "segment-katz-mbe-materials", "text": "The MBE questions used in this study are official multistate bar examination questions from previous administrations of the UBE [65]." } ], "span_id": "span-katz-vor-materials", "supports": "The study components share disclosed exam-item materials and administration lineage." }

Work: em:dossier-source-work:sha256:880443d56eafdb224149d953f7de48e7550d53f814008551895b180ecb4b203b
Edition: em:dossier-edition:sha256:21621ede6a23bb3f98867c0918e5d9808d6dced5ca3f271a3a52e561810f2f7b
Edition digest: sha256:9d10cfedd531e62d96ea27bb78b00a9148e6b5d8135807836c5982e65a42416e
Span: em:dossier-span:sha256:076f2679614413d88a366ceec578a4c96672df24e2fe9dc8afe1242c8c7a5496
Span digest: sha256:9107c310f1bb38d087940d7432c989d2ab81d8e79356bc6a42a6c64bbf2db37e
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree · CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree

GPT-4 Technical Report

Appendix A.3, PDF p. 24

we simply ran these free response questions each only a single time at our best-guess temperature (0.6) and prompt

Work: em:dossier-source-work:sha256:2c6c231a62e11f961467d990d3bd0ad2d4d4568fcd39891c16e6fe4622e9448b
Edition: em:dossier-edition:sha256:859fa96f390685d21c4db9633a6d9c7fec7abe76cdf8a1f8c0c9c82501b193eb
Edition digest: sha256:1703b719a1fad06284c9cde691496e2ee6cc9371ab7808d8abfc50ab3ed0c2f9
Span: em:dossier-span:sha256:8b5e9985c61e0fea14503003ec6422ec54340f4069def5d98d5e1b7147da6ba6
Span digest: sha256:eebbf2c0c2999d72a017e9af6b4799d65577fb0715ac5d85e6ee757e05aae78e
Retrieval: No external retrieval record; repository audit span
License: arXiv non-exclusive distribution license; no Creative Commons license confirmed · arXiv non-exclusive distribution license; no Creative Commons license confirmed

GPT-4 Technical Report

Appendix A.6, PDF p. 25

We ran GPT-4 multiple-choice questions using a model snapshot from March 1, 2023, whereas the free-response questions were run and scored using a non-final model snapshot from February 23, 2023.

Work: em:dossier-source-work:sha256:2c6c231a62e11f961467d990d3bd0ad2d4d4568fcd39891c16e6fe4622e9448b
Edition: em:dossier-edition:sha256:859fa96f390685d21c4db9633a6d9c7fec7abe76cdf8a1f8c0c9c82501b193eb
Edition digest: sha256:1703b719a1fad06284c9cde691496e2ee6cc9371ab7808d8abfc50ab3ed0c2f9
Span: em:dossier-span:sha256:6f6629457faeeb750d379bc919a2030743b0fd07db285521e3a54a46f02ab3ab
Span digest: sha256:9dc5f4260c333122ddb37e7ca7e2f3eecd1d1976475145759b772a7701c58f1e
Retrieval: No external retrieval record; repository audit span
License: arXiv non-exclusive distribution license; no Creative Commons license confirmed · arXiv non-exclusive distribution license; no Creative Commons license confirmed

Appendix for GPT-4 passes the Bar Exam

footnote 4

Best prompt and/or hyperparameter combination on the MBE would push this score to 298 or higher. Here, we report the MBE average of 75.7% which composites to a 297.

Work: em:dossier-source-work:sha256:880443d56eafdb224149d953f7de48e7550d53f814008551895b180ecb4b203b
Edition: em:dossier-edition:sha256:21621ede6a23bb3f98867c0918e5d9808d6dced5ca3f271a3a52e561810f2f7b
Edition digest: sha256:9d10cfedd531e62d96ea27bb78b00a9148e6b5d8135807836c5982e65a42416e
Span: em:dossier-span:sha256:5dd561ae54601a051ef573af4ec859665f4f1d2fb801788a1339de8b16dd44fc
Span digest: sha256:0ce9516e5b95f3b35121d471b8ab8d39b67199ffb40941deb28e07e2a07926ab
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree · CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree

02 The report's 298 and study's approximately 297 are distinct scoring choices within one underlying experiment, not independent performance roots.

Typed relation: support

Relation: relation-assertion-claim-score-discrepancy em:dossier-evidence-relation:sha256:e9d39349fc7f22c3a9130402cf5608a2399cfdd054dfc68f2f80943a58693fea
Proposition: claim-score-discrepancy em:dossier-proposition:sha256:16536a669965de8c6a44bbd1474c6289ed91a1c68f758a1e1c650f572aa7ec19
Assertion: assertion-claim-score-discrepancy em:dossier-assertion:sha256:69abcd1669d34803c874bc6fae4a0c98a6032aaac088d8ee6ee6d7a21b098906
Lineage: lineage-model-performance-root em:dossier-lineage:sha256:c8b81e09b0245e5c5302dd89be2d627f7658d7ab0d5de6d56adef3df0f65e145

Typed dependence

edge-author-social-collaborationlineage-model-performance-root → lineage-model-performance-rootem:dossier-evidence-relation:sha256:b616095462050178a4dedd71e5ad524effaa71d20808eceb206d203f9aae8204accepted_edge_id=edge-author-social-collaboration; accepted_dimension=author-social; from=lineage-model-performance-root; to=lineage-model-performance-root; effects=The vendor report and study are socially linked manifestations, not independent tests.
edge-citation-katz-to-martinezlineage-model-performance-root → lineage-martinez-analysis-rootem:dossier-evidence-relation:sha256:a8c16efd90634c0a152f0fdd7a0a02f8d844004ea05edec24ec4651bf00764ccaccepted_edge_id=edge-citation-katz-to-martinez; accepted_dimension=citation; from=lineage-model-performance-root; to=lineage-martinez-analysis-root; effects=Citation links the analysis lineage to the same reported score.
edge-data-reported-score-reuselineage-model-performance-root → lineage-martinez-analysis-rootem:dossier-evidence-relation:sha256:691d48bc190063d50b6e811c7455ec325b1f29bcc7452301694470e83211188aaccepted_edge_id=edge-data-reported-score-reuse; accepted_dimension=data; from=lineage-model-performance-root; to=lineage-martinez-analysis-root; effects=The re-analysis is not a new model-performance experiment. | Score validation changes warrant, not the participant or model-performance root count.
edge-material-shared-exam-itemslineage-model-performance-root → lineage-model-performance-rootem:dossier-evidence-relation:sha256:8f61eda50599e5c6c226c2dcb7818c8627f23979fd5b51669aa62e5d52f6d110accepted_edge_id=edge-material-shared-exam-items; accepted_dimension=material; from=lineage-model-performance-root; to=lineage-model-performance-root; effects=All score manifestations inherit the same exam-item materials.
edge-method-single-free-response-runlineage-model-performance-root → lineage-model-performance-rootem:dossier-evidence-relation:sha256:1ed5b88123954bc8da25cf9422c2df70e30d3569ddd99213e9cf5590bf8522c3accepted_edge_id=edge-method-single-free-response-run; accepted_dimension=method; from=lineage-model-performance-root; to=lineage-model-performance-root; effects=Repeated documents do not supply repeated-run independence.
edge-model-historical-snapshotslineage-model-performance-root → lineage-model-performance-rootem:dossier-evidence-relation:sha256:e7c0af1eebfe5fa400bcb3369655f40adf482a99cc91a370f6e6a586f847f133accepted_edge_id=edge-model-historical-snapshots; accepted_dimension=model; from=lineage-model-performance-root; to=lineage-model-performance-root; effects=The two snapshots are components of one reported experiment, not two evidence roots.
edge-score-component-compositelineage-model-performance-root → lineage-model-performance-rootem:dossier-evidence-relation:sha256:21278e42f97e114bc1442703ca96eea56725d7bb221a07e5c31f5adfc512b040accepted_edge_id=edge-score-component-composite; accepted_dimension=score; from=lineage-model-performance-root; to=lineage-model-performance-root; effects=297 and 298 are scoring variants within one experiment.

Appendix for GPT-4 passes the Bar Exam

abstract

Graded across the UBE components, in the manner in which a human test-taker would be, GPT-4 scores approximately 297 points

Work: em:dossier-source-work:sha256:880443d56eafdb224149d953f7de48e7550d53f814008551895b180ecb4b203b
Edition: em:dossier-edition:sha256:21621ede6a23bb3f98867c0918e5d9808d6dced5ca3f271a3a52e561810f2f7b
Edition digest: sha256:9d10cfedd531e62d96ea27bb78b00a9148e6b5d8135807836c5982e65a42416e
Span: em:dossier-span:sha256:ee862ec363b34142a0bebb11247959c3457a8982de69653364d7d36f10059703
Span digest: sha256:57920d8649f509df54908d08dfaefb9860e1afc005095c6cd4cb051344dff044
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree · CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree

Appendix for GPT-4 passes the Bar Exam

footnote 4

Best prompt and/or hyperparameter combination on the MBE would push this score to 298 or higher. Here, we report the MBE average of 75.7% which composites to a 297.

Work: em:dossier-source-work:sha256:880443d56eafdb224149d953f7de48e7550d53f814008551895b180ecb4b203b
Edition: em:dossier-edition:sha256:21621ede6a23bb3f98867c0918e5d9808d6dced5ca3f271a3a52e561810f2f7b
Edition digest: sha256:9d10cfedd531e62d96ea27bb78b00a9148e6b5d8135807836c5982e65a42416e
Span: em:dossier-span:sha256:5dd561ae54601a051ef573af4ec859665f4f1d2fb801788a1339de8b16dd44fc
Span digest: sha256:0ce9516e5b95f3b35121d471b8ab8d39b67199ffb40941deb28e07e2a07926ab
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree · CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree

GPT-4 Technical Report

Table 1, PDF p. 5

Uniform Bar Exam (MBE+MEE+MPT) 298 / 400 (~90th)

Work: em:dossier-source-work:sha256:2c6c231a62e11f961467d990d3bd0ad2d4d4568fcd39891c16e6fe4622e9448b
Edition: em:dossier-edition:sha256:859fa96f390685d21c4db9633a6d9c7fec7abe76cdf8a1f8c0c9c82501b193eb
Edition digest: sha256:1703b719a1fad06284c9cde691496e2ee6cc9371ab7808d8abfc50ab3ed0c2f9
Span: em:dossier-span:sha256:959d593b53bd7991549571a95aa96133113958253fa6176851f1fb9bc6203534
Span digest: sha256:9b8313637a2bd3881a98d675793538fbeb77ac8ac64f945ca8ae2d962d3fd79a
Retrieval: No external retrieval record; repository audit span
License: arXiv non-exclusive distribution license; no Creative Commons license confirmed · arXiv non-exclusive distribution license; no Creative Commons license confirmed

GPT-4 Technical Report

Appendix A.1, PDF p. 23

The Uniform Bar Exam was run by our collaborators at CaseText and Stanford CodeX.

Work: em:dossier-source-work:sha256:2c6c231a62e11f961467d990d3bd0ad2d4d4568fcd39891c16e6fe4622e9448b
Edition: em:dossier-edition:sha256:859fa96f390685d21c4db9633a6d9c7fec7abe76cdf8a1f8c0c9c82501b193eb
Edition digest: sha256:1703b719a1fad06284c9cde691496e2ee6cc9371ab7808d8abfc50ab3ed0c2f9
Span: em:dossier-span:sha256:61f2be1da866e5764042a9022c4a0a0eb5d320a0df71f95a24a1a1eb5fd271ac
Span digest: sha256:e0b72dc61e4614a425e991b3fd9641db637b3516311659728f615bed0ea4100c
Retrieval: No external retrieval record; repository audit span
License: arXiv non-exclusive distribution license; no Creative Commons license confirmed · arXiv non-exclusive distribution license; no Creative Commons license confirmed

Appendix for GPT-4 passes the Bar Exam

footnote 5

While we are not fully convinced of the methodological approach taken in some subsequent analysis [78], we do agree that it would be better to consider the raw 297 UBE as falling within a range between 68th and 90th percentile (depending on the precise state and timing of the exam administration).

Work: em:dossier-source-work:sha256:880443d56eafdb224149d953f7de48e7550d53f814008551895b180ecb4b203b
Edition: em:dossier-edition:sha256:21621ede6a23bb3f98867c0918e5d9808d6dced5ca3f271a3a52e561810f2f7b
Edition digest: sha256:9d10cfedd531e62d96ea27bb78b00a9148e6b5d8135807836c5982e65a42416e
Span: em:dossier-span:sha256:477c595f89b6efc4f4fac04d627e65d95daa7929d10f76d3486a96c6c8bf8b06
Span digest: sha256:7e376175b36f8e2809bc74e378c7354858289a689370b70f82df2965d32b9026
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree · CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree

Appendix for GPT-4 passes the Bar Exam

§4(d), Table 7

{ "cells": [ { "cell_id": "cell-katz-gpt4-mbe", "column": "GPT-4", "row": "MBE", "text": "157 points" }, { "cell_id": "cell-katz-gpt4-mee", "column": "GPT-4", "row": "MEE", "text": "84 points" }, { "cell_id": "cell-katz-gpt4-mpt", "column": "GPT-4", "row": "MPT", "text": "56 points" }, { "cell_id": "cell-katz-gpt4-overall", "column": "GPT-4", "row": "overall score", "text": "297 points" } ], "format": "table-cell-transcription", "locator": "\u00a74(d), Table 7", "normalization": "exact-cell-text", "span_id": "span-katz-vor-components", "supports": "The version-of-record component scores sum to 297." }

Work: em:dossier-source-work:sha256:880443d56eafdb224149d953f7de48e7550d53f814008551895b180ecb4b203b
Edition: em:dossier-edition:sha256:21621ede6a23bb3f98867c0918e5d9808d6dced5ca3f271a3a52e561810f2f7b
Edition digest: sha256:9d10cfedd531e62d96ea27bb78b00a9148e6b5d8135807836c5982e65a42416e
Span: em:dossier-span:sha256:293687df5cf4b74d839237414173205591aacf4850103e1621aece4f21fb763b
Span digest: sha256:e6c808d4b4dcd243210518497a0c8788d06ee91d8571c474c13514c2c8afe04b
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree · CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree

percentile_analysis_new.Rmd

introduction, journal p. 584

The paper successfully replicates the MBE score of 158, but highlights several methodological issues in the grading of the MPT + MEE components of the exam, which call into question the validity of the essay score (140).

Work: em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b
Edition: em:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667
Edition digest: sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5
Span: em:dossier-span:sha256:47ce609849ae1ff1224df62ce9b453707dd9b7a93cd4413822c84178eac7f853
Span digest: sha256:11db34f6066f861f338266e98c11852743b727734f62c1a773ae257bf219e609
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed

Appendix for GPT-4 passes the Bar Exam

§3(a), materials

{ "format": "exact-segments", "locator": "\u00a73(a), materials", "normalization": "collapse-whitespace", "segments": [ { "segment_id": "segment-katz-mee-mpt-materials", "text": "For the MEE and the MPT, we collected the most recently released questions from the July 2022 Bar Examination." }, { "segment_id": "segment-katz-mbe-materials", "text": "The MBE questions used in this study are official multistate bar examination questions from previous administrations of the UBE [65]." } ], "span_id": "span-katz-vor-materials", "supports": "The study components share disclosed exam-item materials and administration lineage." }

Work: em:dossier-source-work:sha256:880443d56eafdb224149d953f7de48e7550d53f814008551895b180ecb4b203b
Edition: em:dossier-edition:sha256:21621ede6a23bb3f98867c0918e5d9808d6dced5ca3f271a3a52e561810f2f7b
Edition digest: sha256:9d10cfedd531e62d96ea27bb78b00a9148e6b5d8135807836c5982e65a42416e
Span: em:dossier-span:sha256:076f2679614413d88a366ceec578a4c96672df24e2fe9dc8afe1242c8c7a5496
Span digest: sha256:9107c310f1bb38d087940d7432c989d2ab81d8e79356bc6a42a6c64bbf2db37e
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree · CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree

GPT-4 Technical Report

Appendix A.3, PDF p. 24

we simply ran these free response questions each only a single time at our best-guess temperature (0.6) and prompt

Work: em:dossier-source-work:sha256:2c6c231a62e11f961467d990d3bd0ad2d4d4568fcd39891c16e6fe4622e9448b
Edition: em:dossier-edition:sha256:859fa96f390685d21c4db9633a6d9c7fec7abe76cdf8a1f8c0c9c82501b193eb
Edition digest: sha256:1703b719a1fad06284c9cde691496e2ee6cc9371ab7808d8abfc50ab3ed0c2f9
Span: em:dossier-span:sha256:8b5e9985c61e0fea14503003ec6422ec54340f4069def5d98d5e1b7147da6ba6
Span digest: sha256:eebbf2c0c2999d72a017e9af6b4799d65577fb0715ac5d85e6ee757e05aae78e
Retrieval: No external retrieval record; repository audit span
License: arXiv non-exclusive distribution license; no Creative Commons license confirmed · arXiv non-exclusive distribution license; no Creative Commons license confirmed

GPT-4 Technical Report

Appendix A.6, PDF p. 25

We ran GPT-4 multiple-choice questions using a model snapshot from March 1, 2023, whereas the free-response questions were run and scored using a non-final model snapshot from February 23, 2023.

Work: em:dossier-source-work:sha256:2c6c231a62e11f961467d990d3bd0ad2d4d4568fcd39891c16e6fe4622e9448b
Edition: em:dossier-edition:sha256:859fa96f390685d21c4db9633a6d9c7fec7abe76cdf8a1f8c0c9c82501b193eb
Edition digest: sha256:1703b719a1fad06284c9cde691496e2ee6cc9371ab7808d8abfc50ab3ed0c2f9
Span: em:dossier-span:sha256:6f6629457faeeb750d379bc919a2030743b0fd07db285521e3a54a46f02ab3ab
Span digest: sha256:9dc5f4260c333122ddb37e7ca7e2f3eecd1d1976475145759b772a7701c58f1e
Retrieval: No external retrieval record; repository audit span
License: arXiv non-exclusive distribution license; no Creative Commons license confirmed · arXiv non-exclusive distribution license; no Creative Commons license confirmed

03 A linear interpolation not supplied by the sources places 298 near 89th in the February 2018 chart and 88.6th in the February 2019 chart.

Typed relation: support

Relation: relation-assertion-claim-february-sensitive em:dossier-evidence-relation:sha256:42a1ee6d6d5401c06d4982292f6e68ab82e3eccce9b9367fffc39c67e33c951d
Proposition: claim-february-sensitive em:dossier-proposition:sha256:1034ba4421c1aff2e41f247b73a2bb166fe691ba39953b278845bada50032f33
Assertion: assertion-claim-february-sensitive em:dossier-assertion:sha256:4d19bb08b6c5b6a93562979baf4325fe72ed00c92d854f403e6e3fea4db35665
Lineage: lineage-illinois-comparison-root em:dossier-lineage:sha256:a5b9820aa7d6c9062b89944f259392c039fbe5285b626d4f71d54dfd44f51a20

Typed dependence

edge-benchmark-illinois-chartslineage-illinois-comparison-root → lineage-martinez-analysis-rootem:dossier-evidence-relation:sha256:3bbbc190e0662f0d0cf22c4bf02a5abe6f9e71e95061afd3dc94f535c83fb57aaccepted_edge_id=edge-benchmark-illinois-charts; accepted_dimension=benchmark; from=lineage-illinois-comparison-root; to=lineage-martinez-analysis-root; effects=It is comparison data, not an additional GPT-4 performance root. | Population drift affects rank without changing the model score. | It is a sensitivity input, not a replication of the model run.

Calculation · reviewer sensitivity only: linear interpolation: 89.0; equation="p298 = p290 + ((298 - 290) / 10) * (p300 - p290)"; comparison_population=null; depends_on=[].

Relation: relation-assertion-derive-illinois-feb-2018-298 em:dossier-evidence-relation:sha256:89e465da9ccc872747e2d6c2cf69dbffd57f4c4e3ddd1ec6974b3ce4f3653ecc
Proposition: em:dossier-proposition:sha256:b3eb260778bfb82eeb304b0af6d0f95ae5970cb1e83f64f824eadc453cf0a11f
Assertion: em:dossier-assertion:sha256:f5102c2dfb6527bc325f323f122a10c78a78bc7422ed053fbe00e8edf7d6ab1d

{
  "derivation": {
    "derivation_id": "derive-illinois-feb-2018-298",
    "equation": "p298 = p290 + ((298 - 290) / 10) * (p300 - p290)",
    "input_cell_ids": [
      "cell-illinois-feb-2018-290",
      "cell-illinois-feb-2018-300"
    ],
    "input_span_ids": [
      "span-illinois-feb-2018-anchors"
    ],
    "inputs": {
      "p290": 85.0,
      "p300": 90.0,
      "score": 298
    },
    "method": "reviewer sensitivity only: linear interpolation",
    "result_percentile": 89.0,
    "uncertainty": "Neither Illinois nor OpenAI disclosed this interpolation; it cannot be attributed as the launch method."
  },
  "resolved_input_cells": [
    {
      "cell_id": "cell-illinois-feb-2018-290",
      "percentile": 85,
      "score": 290
    },
    {
      "cell_id": "cell-illinois-feb-2018-300",
      "percentile": 90,
      "score": 300
    }
  ]
}

span-calculation-derive-illinois-feb-2018-298accepted derivation and input cells: derive-illinois-feb-2018-298em:dossier-span:sha256:ba2cde38323123970a17e90771d1f6e21ad66d1ddfa53841749b93e1113694ea · sha256:3037e78d052c9ae7ef8fc027de340daa7e7a1e7488092ce8c1d4ab46cfe1762eem:dossier-edition:sha256:32d3146687af86ae7865d08fbc1636a0c0aa6195e52cb00350ce4b0d525ade2c · sha256:ab330f3d40dfdb5e467da1f66c8f2d318b6b5ed88b770555956d3c5c79151a4bem:dossier-source-work:sha256:bb86be7b07b85a8d09a4fab4b2526a49c82b959ee3fb31bcf33ff7a5e6e27831 · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions
span-illinois-feb-2018-anchorstotal-scale table, rows 300 and 290em:dossier-span:sha256:56a011a09e4e7dd05afbc0309c94d4dcfa1ecf1d6fd3df6a6a6f65204c7ccf03 · sha256:b3b8f789db7571a4fd8ed73e5380f2847aae920172722b9f6f51eec96c62e9ceem:dossier-edition:sha256:9be0e724fc35f74c96e635c6263f8f44ef804c3406c34b268bb710b24228a368 · sha256:44de7bd89cd4aeacee7b38d7077ff659e3baf552cb9b295401ec9b7229c04964em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd · no open license confirmed · no open license confirmed

Calculation · reviewer sensitivity only: linear interpolation: 88.6; equation="p298 = p290 + ((298 - 290) / 10) * (p300 - p290)"; comparison_population=null; depends_on=[].

Relation: relation-assertion-derive-illinois-feb-2019-298 em:dossier-evidence-relation:sha256:eab848ea50c1f7403f46775de29a61ab145b1ef616b6ddedd1befaf47c47d626
Proposition: em:dossier-proposition:sha256:5e417788588abd558e96b1e86a8abbf760107f1b2c5cf87ab39ef6533de01c6f
Assertion: em:dossier-assertion:sha256:2c467a78217c9cfd8e1927f656d2c404085fc79f1544227ad10737e3c0e96bb4

{
  "derivation": {
    "derivation_id": "derive-illinois-feb-2019-298",
    "equation": "p298 = p290 + ((298 - 290) / 10) * (p300 - p290)",
    "input_cell_ids": [
      "cell-illinois-feb-2019-290",
      "cell-illinois-feb-2019-300"
    ],
    "input_span_ids": [
      "span-illinois-feb-2019-anchors"
    ],
    "inputs": {
      "p290": 83.0,
      "p300": 90.0,
      "score": 298
    },
    "method": "reviewer sensitivity only: linear interpolation",
    "result_percentile": 88.6,
    "uncertainty": "Neither Illinois nor OpenAI disclosed this interpolation; it cannot be attributed as the launch method."
  },
  "resolved_input_cells": [
    {
      "cell_id": "cell-illinois-feb-2019-290",
      "percentile": 83,
      "score": 290
    },
    {
      "cell_id": "cell-illinois-feb-2019-300",
      "percentile": 90,
      "score": 300
    }
  ]
}

span-calculation-derive-illinois-feb-2019-298accepted derivation and input cells: derive-illinois-feb-2019-298em:dossier-span:sha256:b16e9a841685191e39868f05b3b6e4ab6adbd109794b3af87845e3609e2bfa8d · sha256:76a57ca09947680e56c24258540370c4538e5e7d69931cb76882834e659fb1f2em:dossier-edition:sha256:32d3146687af86ae7865d08fbc1636a0c0aa6195e52cb00350ce4b0d525ade2c · sha256:ab330f3d40dfdb5e467da1f66c8f2d318b6b5ed88b770555956d3c5c79151a4bem:dossier-source-work:sha256:bb86be7b07b85a8d09a4fab4b2526a49c82b959ee3fb31bcf33ff7a5e6e27831 · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions
span-illinois-feb-2019-anchorstotal-scale table, rows 300 and 290em:dossier-span:sha256:1e2736835309f364d7f8553b6e26a7c9802dce86c720c84fa2097f7e3a755ca3 · sha256:29f7ab38f7ff54e4576aa5b3c6fd310debd0520513f68fc3e21744af073923f8em:dossier-edition:sha256:2c2badc4c75c58ca49007bda1f0c1fcad4b301ecd060bc905bd8a741acd80925 · sha256:91185bd6263d73e5976b5b00f28fb9b62301be0bb0347b1d0a308c295f6a6151em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd · no open license confirmed · no open license confirmed

Calculation · reviewer sensitivity only: linear interpolation: 67.8; equation="p298 = p290 + ((298 - 290) / 10) * (p300 - p290)"; comparison_population=null; depends_on=[].

Relation: relation-assertion-derive-illinois-jul-2018-298 em:dossier-evidence-relation:sha256:76480f3d580abb1b7f822049d98d9d71df986bb9468385f053bc736857135b09
Proposition: em:dossier-proposition:sha256:16d42772ecd2e1cf09e10207fd84bcdd82413184a1ca7e0a6f71222759c24539
Assertion: em:dossier-assertion:sha256:173e036ea146192840c20d9a90986e589e15e92a37c4018264f3dba905e58e4c

{
  "derivation": {
    "derivation_id": "derive-illinois-jul-2018-298",
    "equation": "p298 = p290 + ((298 - 290) / 10) * (p300 - p290)",
    "input_cell_ids": [
      "cell-illinois-jul-2018-290",
      "cell-illinois-jul-2018-300"
    ],
    "input_span_ids": [
      "span-illinois-jul-2018-anchors"
    ],
    "inputs": {
      "p290": 59.0,
      "p300": 70.0,
      "score": 298
    },
    "method": "reviewer sensitivity only: linear interpolation",
    "result_percentile": 67.8,
    "uncertainty": "Neither Illinois nor OpenAI disclosed this interpolation; it cannot be attributed as the launch method."
  },
  "resolved_input_cells": [
    {
      "cell_id": "cell-illinois-jul-2018-290",
      "percentile": 59,
      "score": 290
    },
    {
      "cell_id": "cell-illinois-jul-2018-300",
      "percentile": 70,
      "score": 300
    }
  ]
}

span-calculation-derive-illinois-jul-2018-298accepted derivation and input cells: derive-illinois-jul-2018-298em:dossier-span:sha256:88fbcd4b7b93f462ad98350235a550f2b34f2dbc1a811849f0f40dc7cb03ee18 · sha256:1e02e47e6e3f6a7c2422be3a698ecce3d17899aef055d0f47f2745ec043366f5em:dossier-edition:sha256:32d3146687af86ae7865d08fbc1636a0c0aa6195e52cb00350ce4b0d525ade2c · sha256:ab330f3d40dfdb5e467da1f66c8f2d318b6b5ed88b770555956d3c5c79151a4bem:dossier-source-work:sha256:bb86be7b07b85a8d09a4fab4b2526a49c82b959ee3fb31bcf33ff7a5e6e27831 · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions
span-illinois-jul-2018-anchorstotal-scale table, rows 300 and 290em:dossier-span:sha256:3ec4b97842a7b34ea0a2857a50da7f6a02eb1556351c7ec727b2cccd2743637c · sha256:fe399dcc350821ac8ab546593931f1e34b24d811cad9062126e487a53783a783em:dossier-edition:sha256:2b720aa3b7368249b079f4bc7d3de383efc590510bc33e1ed88db133e9293237 · sha256:a9aa31983924ffec8d0e713ad29fc16a0e466557a57eba5061577afc4698bae6em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd · no open license confirmed · no open license confirmed

Illinois February 2018 Bar Examination Percentile Equivalents

total-scale table, rows 300 and 290

{ "cells": [ { "cell_id": "cell-illinois-feb-2018-300", "percentile": 90, "score": 300 }, { "cell_id": "cell-illinois-feb-2018-290", "percentile": 85, "score": 290 } ], "format": "table-cell-transcription", "locator": "total-scale table, rows 300 and 290", "normalization": "exact-cell-text", "span_id": "span-illinois-feb-2018-anchors", "supports": "The chart brackets 298 but contains no printed 298 row or interpolation rule." }

Work: em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd
Edition: em:dossier-edition:sha256:9be0e724fc35f74c96e635c6263f8f44ef804c3406c34b268bb710b24228a368
Edition digest: sha256:44de7bd89cd4aeacee7b38d7077ff659e3baf552cb9b295401ec9b7229c04964
Span: em:dossier-span:sha256:56a011a09e4e7dd05afbc0309c94d4dcfa1ecf1d6fd3df6a6a6f65204c7ccf03
Span digest: sha256:b3b8f789db7571a4fd8ed73e5380f2847aae920172722b9f6f51eec96c62e9ce
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

Illinois February 2018 Bar Examination Percentile Equivalents

total-scale table, rows 300 and 290

{ "cells": [ { "cell_id": "cell-illinois-feb-2019-300", "percentile": 90, "score": 300 }, { "cell_id": "cell-illinois-feb-2019-290", "percentile": 83, "score": 290 } ], "format": "table-cell-transcription", "locator": "total-scale table, rows 300 and 290", "normalization": "exact-cell-text", "span_id": "span-illinois-feb-2019-anchors", "supports": "This later February chart also brackets 298 but supplies no interpolation rule." }

Work: em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd
Edition: em:dossier-edition:sha256:2c2badc4c75c58ca49007bda1f0c1fcad4b301ecd060bc905bd8a741acd80925
Edition digest: sha256:91185bd6263d73e5976b5b00f28fb9b62301be0bb0347b1d0a308c295f6a6151
Span: em:dossier-span:sha256:1e2736835309f364d7f8553b6e26a7c9802dce86c720c84fa2097f7e3a755ca3
Span digest: sha256:29f7ab38f7ff54e4576aa5b3c6fd310debd0520513f68fc3e21744af073923f8
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

Illinois February 2018 Bar Examination Percentile Equivalents

total-scale table, rows 300 and 290

{ "cells": [ { "cell_id": "cell-illinois-jul-2018-300", "percentile": 70, "score": 300 }, { "cell_id": "cell-illinois-jul-2018-290", "percentile": 59, "score": 290 } ], "format": "table-cell-transcription", "locator": "total-scale table, rows 300 and 290", "normalization": "exact-cell-text", "span_id": "span-illinois-jul-2018-anchors", "supports": "The July chart places the same score region far below the February chart." }

Work: em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd
Edition: em:dossier-edition:sha256:2b720aa3b7368249b079f4bc7d3de383efc590510bc33e1ed88db133e9293237
Edition digest: sha256:a9aa31983924ffec8d0e713ad29fc16a0e466557a57eba5061577afc4698bae6
Span: em:dossier-span:sha256:3ec4b97842a7b34ea0a2857a50da7f6a02eb1556351c7ec727b2cccd2743637c
Span digest: sha256:fe399dcc350821ac8ab546593931f1e34b24d811cad9062126e487a53783a783
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

04 The same disclosed linear interpolation places 298 near 67.8th in the July 2018 Illinois chart.

Typed relation: support

Relation: relation-assertion-claim-july-sensitive em:dossier-evidence-relation:sha256:73716abf204fa15d841ff6b95c2dfbe303e3fcb4e06c26923a8fa5a618593415
Proposition: claim-july-sensitive em:dossier-proposition:sha256:c03843891cd010c53ddbe03e8d0128baeeccd8dedde381d2aaf3ed9a8bf08752
Assertion: assertion-claim-july-sensitive em:dossier-assertion:sha256:8feb34c96f4f746c4cea970e4ced3e891be6c2faa4d74e8370abed8ab69e8371
Lineage: lineage-illinois-comparison-root em:dossier-lineage:sha256:a5b9820aa7d6c9062b89944f259392c039fbe5285b626d4f71d54dfd44f51a20

Typed dependence

edge-benchmark-illinois-chartslineage-illinois-comparison-root → lineage-martinez-analysis-rootem:dossier-evidence-relation:sha256:3bbbc190e0662f0d0cf22c4bf02a5abe6f9e71e95061afd3dc94f535c83fb57aaccepted_edge_id=edge-benchmark-illinois-charts; accepted_dimension=benchmark; from=lineage-illinois-comparison-root; to=lineage-martinez-analysis-root; effects=It is comparison data, not an additional GPT-4 performance root. | Population drift affects rank without changing the model score. | It is a sensitivity input, not a replication of the model run.

Calculation · reviewer sensitivity only: linear interpolation: 89.0; equation="p298 = p290 + ((298 - 290) / 10) * (p300 - p290)"; comparison_population=null; depends_on=[].

Relation: relation-assertion-derive-illinois-feb-2018-298 em:dossier-evidence-relation:sha256:89e465da9ccc872747e2d6c2cf69dbffd57f4c4e3ddd1ec6974b3ce4f3653ecc
Proposition: em:dossier-proposition:sha256:b3eb260778bfb82eeb304b0af6d0f95ae5970cb1e83f64f824eadc453cf0a11f
Assertion: em:dossier-assertion:sha256:f5102c2dfb6527bc325f323f122a10c78a78bc7422ed053fbe00e8edf7d6ab1d

{
  "derivation": {
    "derivation_id": "derive-illinois-feb-2018-298",
    "equation": "p298 = p290 + ((298 - 290) / 10) * (p300 - p290)",
    "input_cell_ids": [
      "cell-illinois-feb-2018-290",
      "cell-illinois-feb-2018-300"
    ],
    "input_span_ids": [
      "span-illinois-feb-2018-anchors"
    ],
    "inputs": {
      "p290": 85.0,
      "p300": 90.0,
      "score": 298
    },
    "method": "reviewer sensitivity only: linear interpolation",
    "result_percentile": 89.0,
    "uncertainty": "Neither Illinois nor OpenAI disclosed this interpolation; it cannot be attributed as the launch method."
  },
  "resolved_input_cells": [
    {
      "cell_id": "cell-illinois-feb-2018-290",
      "percentile": 85,
      "score": 290
    },
    {
      "cell_id": "cell-illinois-feb-2018-300",
      "percentile": 90,
      "score": 300
    }
  ]
}

span-calculation-derive-illinois-feb-2018-298accepted derivation and input cells: derive-illinois-feb-2018-298em:dossier-span:sha256:ba2cde38323123970a17e90771d1f6e21ad66d1ddfa53841749b93e1113694ea · sha256:3037e78d052c9ae7ef8fc027de340daa7e7a1e7488092ce8c1d4ab46cfe1762eem:dossier-edition:sha256:32d3146687af86ae7865d08fbc1636a0c0aa6195e52cb00350ce4b0d525ade2c · sha256:ab330f3d40dfdb5e467da1f66c8f2d318b6b5ed88b770555956d3c5c79151a4bem:dossier-source-work:sha256:bb86be7b07b85a8d09a4fab4b2526a49c82b959ee3fb31bcf33ff7a5e6e27831 · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions
span-illinois-feb-2018-anchorstotal-scale table, rows 300 and 290em:dossier-span:sha256:56a011a09e4e7dd05afbc0309c94d4dcfa1ecf1d6fd3df6a6a6f65204c7ccf03 · sha256:b3b8f789db7571a4fd8ed73e5380f2847aae920172722b9f6f51eec96c62e9ceem:dossier-edition:sha256:9be0e724fc35f74c96e635c6263f8f44ef804c3406c34b268bb710b24228a368 · sha256:44de7bd89cd4aeacee7b38d7077ff659e3baf552cb9b295401ec9b7229c04964em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd · no open license confirmed · no open license confirmed

Calculation · reviewer sensitivity only: linear interpolation: 88.6; equation="p298 = p290 + ((298 - 290) / 10) * (p300 - p290)"; comparison_population=null; depends_on=[].

Relation: relation-assertion-derive-illinois-feb-2019-298 em:dossier-evidence-relation:sha256:eab848ea50c1f7403f46775de29a61ab145b1ef616b6ddedd1befaf47c47d626
Proposition: em:dossier-proposition:sha256:5e417788588abd558e96b1e86a8abbf760107f1b2c5cf87ab39ef6533de01c6f
Assertion: em:dossier-assertion:sha256:2c467a78217c9cfd8e1927f656d2c404085fc79f1544227ad10737e3c0e96bb4

{
  "derivation": {
    "derivation_id": "derive-illinois-feb-2019-298",
    "equation": "p298 = p290 + ((298 - 290) / 10) * (p300 - p290)",
    "input_cell_ids": [
      "cell-illinois-feb-2019-290",
      "cell-illinois-feb-2019-300"
    ],
    "input_span_ids": [
      "span-illinois-feb-2019-anchors"
    ],
    "inputs": {
      "p290": 83.0,
      "p300": 90.0,
      "score": 298
    },
    "method": "reviewer sensitivity only: linear interpolation",
    "result_percentile": 88.6,
    "uncertainty": "Neither Illinois nor OpenAI disclosed this interpolation; it cannot be attributed as the launch method."
  },
  "resolved_input_cells": [
    {
      "cell_id": "cell-illinois-feb-2019-290",
      "percentile": 83,
      "score": 290
    },
    {
      "cell_id": "cell-illinois-feb-2019-300",
      "percentile": 90,
      "score": 300
    }
  ]
}

span-calculation-derive-illinois-feb-2019-298accepted derivation and input cells: derive-illinois-feb-2019-298em:dossier-span:sha256:b16e9a841685191e39868f05b3b6e4ab6adbd109794b3af87845e3609e2bfa8d · sha256:76a57ca09947680e56c24258540370c4538e5e7d69931cb76882834e659fb1f2em:dossier-edition:sha256:32d3146687af86ae7865d08fbc1636a0c0aa6195e52cb00350ce4b0d525ade2c · sha256:ab330f3d40dfdb5e467da1f66c8f2d318b6b5ed88b770555956d3c5c79151a4bem:dossier-source-work:sha256:bb86be7b07b85a8d09a4fab4b2526a49c82b959ee3fb31bcf33ff7a5e6e27831 · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions
span-illinois-feb-2019-anchorstotal-scale table, rows 300 and 290em:dossier-span:sha256:1e2736835309f364d7f8553b6e26a7c9802dce86c720c84fa2097f7e3a755ca3 · sha256:29f7ab38f7ff54e4576aa5b3c6fd310debd0520513f68fc3e21744af073923f8em:dossier-edition:sha256:2c2badc4c75c58ca49007bda1f0c1fcad4b301ecd060bc905bd8a741acd80925 · sha256:91185bd6263d73e5976b5b00f28fb9b62301be0bb0347b1d0a308c295f6a6151em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd · no open license confirmed · no open license confirmed

Calculation · reviewer sensitivity only: linear interpolation: 67.8; equation="p298 = p290 + ((298 - 290) / 10) * (p300 - p290)"; comparison_population=null; depends_on=[].

Relation: relation-assertion-derive-illinois-jul-2018-298 em:dossier-evidence-relation:sha256:76480f3d580abb1b7f822049d98d9d71df986bb9468385f053bc736857135b09
Proposition: em:dossier-proposition:sha256:16d42772ecd2e1cf09e10207fd84bcdd82413184a1ca7e0a6f71222759c24539
Assertion: em:dossier-assertion:sha256:173e036ea146192840c20d9a90986e589e15e92a37c4018264f3dba905e58e4c

{
  "derivation": {
    "derivation_id": "derive-illinois-jul-2018-298",
    "equation": "p298 = p290 + ((298 - 290) / 10) * (p300 - p290)",
    "input_cell_ids": [
      "cell-illinois-jul-2018-290",
      "cell-illinois-jul-2018-300"
    ],
    "input_span_ids": [
      "span-illinois-jul-2018-anchors"
    ],
    "inputs": {
      "p290": 59.0,
      "p300": 70.0,
      "score": 298
    },
    "method": "reviewer sensitivity only: linear interpolation",
    "result_percentile": 67.8,
    "uncertainty": "Neither Illinois nor OpenAI disclosed this interpolation; it cannot be attributed as the launch method."
  },
  "resolved_input_cells": [
    {
      "cell_id": "cell-illinois-jul-2018-290",
      "percentile": 59,
      "score": 290
    },
    {
      "cell_id": "cell-illinois-jul-2018-300",
      "percentile": 70,
      "score": 300
    }
  ]
}

span-calculation-derive-illinois-jul-2018-298accepted derivation and input cells: derive-illinois-jul-2018-298em:dossier-span:sha256:88fbcd4b7b93f462ad98350235a550f2b34f2dbc1a811849f0f40dc7cb03ee18 · sha256:1e02e47e6e3f6a7c2422be3a698ecce3d17899aef055d0f47f2745ec043366f5em:dossier-edition:sha256:32d3146687af86ae7865d08fbc1636a0c0aa6195e52cb00350ce4b0d525ade2c · sha256:ab330f3d40dfdb5e467da1f66c8f2d318b6b5ed88b770555956d3c5c79151a4bem:dossier-source-work:sha256:bb86be7b07b85a8d09a4fab4b2526a49c82b959ee3fb31bcf33ff7a5e6e27831 · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions
span-illinois-jul-2018-anchorstotal-scale table, rows 300 and 290em:dossier-span:sha256:3ec4b97842a7b34ea0a2857a50da7f6a02eb1556351c7ec727b2cccd2743637c · sha256:fe399dcc350821ac8ab546593931f1e34b24d811cad9062126e487a53783a783em:dossier-edition:sha256:2b720aa3b7368249b079f4bc7d3de383efc590510bc33e1ed88db133e9293237 · sha256:a9aa31983924ffec8d0e713ad29fc16a0e466557a57eba5061577afc4698bae6em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd · no open license confirmed · no open license confirmed

Illinois February 2018 Bar Examination Percentile Equivalents

total-scale table, rows 300 and 290

{ "cells": [ { "cell_id": "cell-illinois-jul-2018-300", "percentile": 70, "score": 300 }, { "cell_id": "cell-illinois-jul-2018-290", "percentile": 59, "score": 290 } ], "format": "table-cell-transcription", "locator": "total-scale table, rows 300 and 290", "normalization": "exact-cell-text", "span_id": "span-illinois-jul-2018-anchors", "supports": "The July chart places the same score region far below the February chart." }

Work: em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd
Edition: em:dossier-edition:sha256:2b720aa3b7368249b079f4bc7d3de383efc590510bc33e1ed88db133e9293237
Edition digest: sha256:a9aa31983924ffec8d0e713ad29fc16a0e466557a57eba5061577afc4698bae6
Span: em:dossier-span:sha256:3ec4b97842a7b34ea0a2857a50da7f6a02eb1556351c7ec727b2cccd2743637c
Span digest: sha256:fe399dcc350821ac8ab546593931f1e34b24d811cad9062126e487a53783a783
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

Illinois February 2018 Bar Examination Percentile Equivalents

total-scale table, rows 300 and 290

{ "cells": [ { "cell_id": "cell-illinois-feb-2018-300", "percentile": 90, "score": 300 }, { "cell_id": "cell-illinois-feb-2018-290", "percentile": 85, "score": 290 } ], "format": "table-cell-transcription", "locator": "total-scale table, rows 300 and 290", "normalization": "exact-cell-text", "span_id": "span-illinois-feb-2018-anchors", "supports": "The chart brackets 298 but contains no printed 298 row or interpolation rule." }

Work: em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd
Edition: em:dossier-edition:sha256:9be0e724fc35f74c96e635c6263f8f44ef804c3406c34b268bb710b24228a368
Edition digest: sha256:44de7bd89cd4aeacee7b38d7077ff659e3baf552cb9b295401ec9b7229c04964
Span: em:dossier-span:sha256:56a011a09e4e7dd05afbc0309c94d4dcfa1ecf1d6fd3df6a6a6f65204c7ccf03
Span digest: sha256:b3b8f789db7571a4fd8ed73e5380f2847aae920172722b9f6f51eec96c62e9ce
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

Illinois February 2018 Bar Examination Percentile Equivalents

total-scale table, rows 300 and 290

{ "cells": [ { "cell_id": "cell-illinois-feb-2019-300", "percentile": 90, "score": 300 }, { "cell_id": "cell-illinois-feb-2019-290", "percentile": 83, "score": 290 } ], "format": "table-cell-transcription", "locator": "total-scale table, rows 300 and 290", "normalization": "exact-cell-text", "span_id": "span-illinois-feb-2019-anchors", "supports": "This later February chart also brackets 298 but supplies no interpolation rule." }

Work: em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd
Edition: em:dossier-edition:sha256:2c2badc4c75c58ca49007bda1f0c1fcad4b301ecd060bc905bd8a741acd80925
Edition digest: sha256:91185bd6263d73e5976b5b00f28fb9b62301be0bb0347b1d0a308c295f6a6151
Span: em:dossier-span:sha256:1e2736835309f364d7f8553b6e26a7c9802dce86c720c84fa2097f7e3a755ca3
Span digest: sha256:29f7ab38f7ff54e4576aa5b3c6fd310debd0520513f68fc3e21744af073923f8
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

05 Under Martínez's modeled assumptions, 298 is approximately 62nd percentile among first-time takers.

Typed relation: support

Relation: relation-assertion-claim-martinez-first-time em:dossier-evidence-relation:sha256:9eab2effd3cc904155efeba8e0f0fcc8a1c651a180827275569eafefd16b4193
Proposition: claim-martinez-first-time em:dossier-proposition:sha256:fe66108f071fdcc1e63e50df325d22f059b385b733de0db6a9c2b4e8a3357474
Assertion: assertion-claim-martinez-first-time em:dossier-assertion:sha256:1e01cf8d95dfde4cf053070bfdcc291f54e0cc8280922b2f323d41cb231d213d
Lineage: lineage-martinez-analysis-root em:dossier-lineage:sha256:e6057f0dbfa37d0264e471e67a3941297a56467e67cad715d0eb37490d320cdd

Typed dependence

edge-benchmark-illinois-chartslineage-illinois-comparison-root → lineage-martinez-analysis-rootem:dossier-evidence-relation:sha256:3bbbc190e0662f0d0cf22c4bf02a5abe6f9e71e95061afd3dc94f535c83fb57aaccepted_edge_id=edge-benchmark-illinois-charts; accepted_dimension=benchmark; from=lineage-illinois-comparison-root; to=lineage-martinez-analysis-root; effects=It is comparison data, not an additional GPT-4 performance root. | Population drift affects rank without changing the model score. | It is a sensitivity input, not a replication of the model run.
edge-citation-katz-to-martinezlineage-model-performance-root → lineage-martinez-analysis-rootem:dossier-evidence-relation:sha256:a8c16efd90634c0a152f0fdd7a0a02f8d844004ea05edec24ec4651bf00764ccaccepted_edge_id=edge-citation-katz-to-martinez; accepted_dimension=citation; from=lineage-model-performance-root; to=lineage-martinez-analysis-root; effects=Citation links the analysis lineage to the same reported score.
edge-comparison-class-first-time-passers--1lineage-ncbe-comparison-root → lineage-martinez-analysis-rootem:dossier-evidence-relation:sha256:c26568911130747cbc372db3bcb7c6a64cecaa0134fb60ab0def98d7ceb621e4accepted_edge_id=edge-comparison-class-first-time-passers; accepted_dimension=comparison-class; from=lineage-ncbe-comparison-root,lineage-new-york-pass-rate-root; to=lineage-martinez-analysis-root; effects=Comparison populations are not interchangeable. | Administration mix can change percentile without new model evidence. | It parameterizes the comparison model rather than testing GPT-4. | The threshold defines a modeled comparator boundary.
edge-comparison-class-first-time-passers--2lineage-new-york-pass-rate-root → lineage-martinez-analysis-rootem:dossier-evidence-relation:sha256:0b7e8f83a90479b952cdbf35c12046e8810bf312a6f413158f9d4567bc45849baccepted_edge_id=edge-comparison-class-first-time-passers; accepted_dimension=comparison-class; from=lineage-ncbe-comparison-root,lineage-new-york-pass-rate-root; to=lineage-martinez-analysis-root; effects=Comparison populations are not interchangeable. | Administration mix can change percentile without new model evidence. | It parameterizes the comparison model rather than testing GPT-4. | The threshold defines a modeled comparator boundary.
edge-data-reported-score-reuselineage-model-performance-root → lineage-martinez-analysis-rootem:dossier-evidence-relation:sha256:691d48bc190063d50b6e811c7455ec325b1f29bcc7452301694470e83211188aaccepted_edge_id=edge-data-reported-score-reuse; accepted_dimension=data; from=lineage-model-performance-root; to=lineage-martinez-analysis-root; effects=The re-analysis is not a new model-performance experiment. | Score validation changes warrant, not the participant or model-performance root count.
edge-derivation-comparison-inputs--1lineage-ncbe-comparison-root → lineage-martinez-analysis-rootem:dossier-evidence-relation:sha256:8e1e29f851db71dfa10a5e236ce568d68a73d9c9dcc9e57be569669606ec66a7accepted_edge_id=edge-derivation-comparison-inputs; accepted_dimension=derivation; from=lineage-ncbe-comparison-root,lineage-new-york-pass-rate-root; to=lineage-martinez-analysis-root; effects=The value is a modeled input, not a GPT-4 observation. | Executable transformations do not create new source or performance roots.
edge-derivation-comparison-inputs--2lineage-new-york-pass-rate-root → lineage-martinez-analysis-rootem:dossier-evidence-relation:sha256:4cb023d071772f2c5de160eda970a23d215bd0e51de0ebef48513742b7b92ab4accepted_edge_id=edge-derivation-comparison-inputs; accepted_dimension=derivation; from=lineage-ncbe-comparison-root,lineage-new-york-pass-rate-root; to=lineage-martinez-analysis-root; effects=The value is a modeled input, not a GPT-4 observation. | Executable transformations do not create new source or performance roots.

Calculation · normal CDF at essay score 140 using assumed MBE distribution: 41.520892709392534; equation=null; comparison_population="modeled first-time essay scores"; depends_on=["derive-martinez-parameters"].

Relation: relation-assertion-derive-martinez-first-time-essay em:dossier-evidence-relation:sha256:13748298644aa578769bbabfff2cef7491504ee79bf1d180ece2b059135ed552
Proposition: em:dossier-proposition:sha256:8b4297b5104a65c7f8bddf8b32ed22ff861e9c48bfd3f310cd2004f0422891a3
Assertion: em:dossier-assertion:sha256:cd35ba2db639224fb309527b6345a677d99b6477c0af28d3b1780b3b760db33e

{
  "derivation": {
    "comparison_population": "modeled first-time essay scores",
    "depends_on": [
      "derive-martinez-parameters"
    ],
    "derivation_id": "derive-martinez-first-time-essay",
    "method": "normal CDF at essay score 140 using assumed MBE distribution",
    "result_percentile": 41.520892709392534
  },
  "resolved_input_cells": []
}

span-calculation-derive-martinez-first-time-essayaccepted derivation and input cells: derive-martinez-first-time-essayem:dossier-span:sha256:7fe07e6a8c3bfc7145de80e008268fc0c0e4c52008cdcc2c56229e50818ae891 · sha256:a9c27ee78ab735acd0cc44a48abf93543d47b7e31106e8b749ad9acd73d94272em:dossier-edition:sha256:32d3146687af86ae7865d08fbc1636a0c0aa6195e52cb00350ce4b0d525ade2c · sha256:ab330f3d40dfdb5e467da1f66c8f2d318b6b5ed88b770555956d3c5c79151a4bem:dossier-source-work:sha256:bb86be7b07b85a8d09a4fab4b2526a49c82b959ee3fb31bcf33ff7a5e6e27831 · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions
span-martinez-mean-assumption§3.1, journal p. 588em:dossier-span:sha256:ecc711638dfc4b30115783c7b8d7f32b292b8c723a36f8dfbf421e41923b9461 · sha256:d292fbd6a0a2c16559d69c651b273374cef4c7383b7c2605b97f85381bf8c62dem:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667 · sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-july-mbe-distributionlines 55-58em:dossier-span:sha256:c93f04ab5c92ae7955b75278cff1f0392210fc5462823dbe30cba42b5ffbab38 · sha256:9912e2d4f1e369b4f9529d27b6d1fef5936814f0c6151a9fe9f675af8c375e0cem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-ube-sdlines 104-112em:dossier-span:sha256:7c297fee53ebcae4ddc2b4bd2301d9742b570914c364b3993890c92c3ab42d83 · sha256:29d389334aa2b12045b1d9febb630b66d80d832615d0764534e4e0a2ac49b55bem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-ncbe-ny-cutoff-2022Minimum Passing UBE Score by Jurisdiction in 2022, row 266em:dossier-span:sha256:160241d336f3cc06be5b4d52f6ac9229563916a9ca197dbbacb968b7c60981f8 · sha256:8e029f660745d0cb0de2acd77e8d6e8c8f9c544d1ba560c3250bc30861ad653bem:dossier-edition:sha256:2000492e8e5040cd8f5eaa216e7fc1d4d21d4f58afbfc6af5a218f4f53333081 · sha256:b55af410679b66b8a0057d5f8dd17baec7985675c94d7af44818f5c4c418cfb7em:dossier-source-work:sha256:324982bb2b90295b84f9e309ab84ca9a46cd0f1be9e9e3079a18bcc20bf58e63 · no open license confirmed · no open license confirmed
span-ny-first-timers-2022FIRST TIMERS, ALL Candidates, Combined 2022em:dossier-span:sha256:468763dc341321dd638ef9a08f6606ac5e02806a4086e4b0ba7bc05400c1d703 · sha256:e55680bc3d09d5c2968a8449c122db232a89c5f89b29dc9944b69c2b1eeeee11em:dossier-edition:sha256:10bbf9ff899ba3ad12de154e0ab9ea7cc3f34c0f2263e79951e26539911bec5f · sha256:bbdcb28fb38b0d7d03ddb4e4bcfbcca170d2443491a180fee0978c7510cdb74fem:dossier-source-work:sha256:51aa3262776f5da4a8ef12ab7f49366ef761951213e3a38f519a0a6b5d6534f1 · no open license confirmed · no open license confirmed
span-reshetar-first-time-meanparagraph beginning 'The numbers from 2019'em:dossier-span:sha256:6f0efeaf8fa67729f3f75793e1b64ce9961bb3463e5acc9d9c4a1d5aeb588520 · sha256:1452bea6f3158b2b58d90800d02a6cf2348935e6a194e07ab22efc88c5f26302em:dossier-edition:sha256:852a2ad23d9f6f760f0f21c4147eb6dd1b3c936ea9704e3dc3f743de193e4e49 · sha256:460f8d22d10526ee2a92e1fb4045e1cabacb0dc4a668e142a91a30592ccf0ed2em:dossier-source-work:sha256:802fad21dc2e69f5baff90d345e036b898bf958351af8eae39f346d23e005a12 · no open license confirmed · no open license confirmed

Calculation · normal CDF at MBE score 158: 78.82324690739395; equation=null; comparison_population="modeled first-time MBE takers"; depends_on=["derive-martinez-parameters"].

Relation: relation-assertion-derive-martinez-first-time-mbe em:dossier-evidence-relation:sha256:e7b1110a59e6ed3c7a511c4e1e77aa7664a34e076f572963e8d3790a981334bc
Proposition: em:dossier-proposition:sha256:added6c64552253aad1f336b1c4d88de6dd6172647d50fa5bddd3820feecf458
Assertion: em:dossier-assertion:sha256:7df08eee7d48ee98980dc48dcd4257c04f6a03ee5db70b69f16762e4122880ff

{
  "derivation": {
    "comparison_population": "modeled first-time MBE takers",
    "depends_on": [
      "derive-martinez-parameters"
    ],
    "derivation_id": "derive-martinez-first-time-mbe",
    "method": "normal CDF at MBE score 158",
    "result_percentile": 78.82324690739395
  },
  "resolved_input_cells": []
}

span-calculation-derive-martinez-first-time-mbeaccepted derivation and input cells: derive-martinez-first-time-mbeem:dossier-span:sha256:46a2e94c6b7fa7049cb7e26fb0def120ac9445a8bd95c1b21478c0db1e94424a · sha256:0222f9c217d52a28eaeb28caf33d560864fd14ba99c48d7ebd7fa0aa4fdba995em:dossier-edition:sha256:32d3146687af86ae7865d08fbc1636a0c0aa6195e52cb00350ce4b0d525ade2c · sha256:ab330f3d40dfdb5e467da1f66c8f2d318b6b5ed88b770555956d3c5c79151a4bem:dossier-source-work:sha256:bb86be7b07b85a8d09a4fab4b2526a49c82b959ee3fb31bcf33ff7a5e6e27831 · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions
span-martinez-mean-assumption§3.1, journal p. 588em:dossier-span:sha256:ecc711638dfc4b30115783c7b8d7f32b292b8c723a36f8dfbf421e41923b9461 · sha256:d292fbd6a0a2c16559d69c651b273374cef4c7383b7c2605b97f85381bf8c62dem:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667 · sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-july-mbe-distributionlines 55-58em:dossier-span:sha256:c93f04ab5c92ae7955b75278cff1f0392210fc5462823dbe30cba42b5ffbab38 · sha256:9912e2d4f1e369b4f9529d27b6d1fef5936814f0c6151a9fe9f675af8c375e0cem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-ube-sdlines 104-112em:dossier-span:sha256:7c297fee53ebcae4ddc2b4bd2301d9742b570914c364b3993890c92c3ab42d83 · sha256:29d389334aa2b12045b1d9febb630b66d80d832615d0764534e4e0a2ac49b55bem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-ncbe-ny-cutoff-2022Minimum Passing UBE Score by Jurisdiction in 2022, row 266em:dossier-span:sha256:160241d336f3cc06be5b4d52f6ac9229563916a9ca197dbbacb968b7c60981f8 · sha256:8e029f660745d0cb0de2acd77e8d6e8c8f9c544d1ba560c3250bc30861ad653bem:dossier-edition:sha256:2000492e8e5040cd8f5eaa216e7fc1d4d21d4f58afbfc6af5a218f4f53333081 · sha256:b55af410679b66b8a0057d5f8dd17baec7985675c94d7af44818f5c4c418cfb7em:dossier-source-work:sha256:324982bb2b90295b84f9e309ab84ca9a46cd0f1be9e9e3079a18bcc20bf58e63 · no open license confirmed · no open license confirmed
span-ny-first-timers-2022FIRST TIMERS, ALL Candidates, Combined 2022em:dossier-span:sha256:468763dc341321dd638ef9a08f6606ac5e02806a4086e4b0ba7bc05400c1d703 · sha256:e55680bc3d09d5c2968a8449c122db232a89c5f89b29dc9944b69c2b1eeeee11em:dossier-edition:sha256:10bbf9ff899ba3ad12de154e0ab9ea7cc3f34c0f2263e79951e26539911bec5f · sha256:bbdcb28fb38b0d7d03ddb4e4bcfbcca170d2443491a180fee0978c7510cdb74fem:dossier-source-work:sha256:51aa3262776f5da4a8ef12ab7f49366ef761951213e3a38f519a0a6b5d6534f1 · no open license confirmed · no open license confirmed
span-reshetar-first-time-meanparagraph beginning 'The numbers from 2019'em:dossier-span:sha256:6f0efeaf8fa67729f3f75793e1b64ce9961bb3463e5acc9d9c4a1d5aeb588520 · sha256:1452bea6f3158b2b58d90800d02a6cf2348935e6a194e07ab22efc88c5f26302em:dossier-edition:sha256:852a2ad23d9f6f760f0f21c4147eb6dd1b3c936ea9704e3dc3f743de193e4e49 · sha256:460f8d22d10526ee2a92e1fb4045e1cabacb0dc4a668e142a91a30592ccf0ed2em:dossier-source-work:sha256:802fad21dc2e69f5baff90d345e036b898bf958351af8eae39f346d23e005a12 · no open license confirmed · no open license confirmed

Calculation · normal CDF at score 298: 61.60252541656707; equation="100 * Phi((298 - 287.6) / derived_ube_sd)"; comparison_population="modeled first-time UBE takers"; depends_on=["derive-martinez-parameters"].

Relation: relation-assertion-derive-martinez-first-time-ube em:dossier-evidence-relation:sha256:a2c43a47e3f6b92d24687c9aa8fc109e6c1395031b0d99748789cacc006a0718
Proposition: em:dossier-proposition:sha256:e239947e9f182874d45854038dd31c8374fadebae4638ef0e25f91f286fe7202
Assertion: em:dossier-assertion:sha256:a5fe847c3d60f4185c2335a2160eb3e8223fb2e1a8e2174b611efd1cc9f442d0

{
  "derivation": {
    "comparison_population": "modeled first-time UBE takers",
    "depends_on": [
      "derive-martinez-parameters"
    ],
    "derivation_id": "derive-martinez-first-time-ube",
    "equation": "100 * Phi((298 - 287.6) / derived_ube_sd)",
    "method": "normal CDF at score 298",
    "result_percentile": 61.60252541656707
  },
  "resolved_input_cells": []
}

span-calculation-derive-martinez-first-time-ubeaccepted derivation and input cells: derive-martinez-first-time-ubeem:dossier-span:sha256:282eac343e251f80212871bfab6bbad216ef94e8989d5f56387cc19a79d72aca · sha256:95f5718ef7d21ae6fb0218ba919638f6bee7e0a849abfab1144857c83941d8c2em:dossier-edition:sha256:32d3146687af86ae7865d08fbc1636a0c0aa6195e52cb00350ce4b0d525ade2c · sha256:ab330f3d40dfdb5e467da1f66c8f2d318b6b5ed88b770555956d3c5c79151a4bem:dossier-source-work:sha256:bb86be7b07b85a8d09a4fab4b2526a49c82b959ee3fb31bcf33ff7a5e6e27831 · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions
span-martinez-mean-assumption§3.1, journal p. 588em:dossier-span:sha256:ecc711638dfc4b30115783c7b8d7f32b292b8c723a36f8dfbf421e41923b9461 · sha256:d292fbd6a0a2c16559d69c651b273374cef4c7383b7c2605b97f85381bf8c62dem:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667 · sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-july-mbe-distributionlines 55-58em:dossier-span:sha256:c93f04ab5c92ae7955b75278cff1f0392210fc5462823dbe30cba42b5ffbab38 · sha256:9912e2d4f1e369b4f9529d27b6d1fef5936814f0c6151a9fe9f675af8c375e0cem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-ube-sdlines 104-112em:dossier-span:sha256:7c297fee53ebcae4ddc2b4bd2301d9742b570914c364b3993890c92c3ab42d83 · sha256:29d389334aa2b12045b1d9febb630b66d80d832615d0764534e4e0a2ac49b55bem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-ncbe-ny-cutoff-2022Minimum Passing UBE Score by Jurisdiction in 2022, row 266em:dossier-span:sha256:160241d336f3cc06be5b4d52f6ac9229563916a9ca197dbbacb968b7c60981f8 · sha256:8e029f660745d0cb0de2acd77e8d6e8c8f9c544d1ba560c3250bc30861ad653bem:dossier-edition:sha256:2000492e8e5040cd8f5eaa216e7fc1d4d21d4f58afbfc6af5a218f4f53333081 · sha256:b55af410679b66b8a0057d5f8dd17baec7985675c94d7af44818f5c4c418cfb7em:dossier-source-work:sha256:324982bb2b90295b84f9e309ab84ca9a46cd0f1be9e9e3079a18bcc20bf58e63 · no open license confirmed · no open license confirmed
span-ny-first-timers-2022FIRST TIMERS, ALL Candidates, Combined 2022em:dossier-span:sha256:468763dc341321dd638ef9a08f6606ac5e02806a4086e4b0ba7bc05400c1d703 · sha256:e55680bc3d09d5c2968a8449c122db232a89c5f89b29dc9944b69c2b1eeeee11em:dossier-edition:sha256:10bbf9ff899ba3ad12de154e0ab9ea7cc3f34c0f2263e79951e26539911bec5f · sha256:bbdcb28fb38b0d7d03ddb4e4bcfbcca170d2443491a180fee0978c7510cdb74fem:dossier-source-work:sha256:51aa3262776f5da4a8ef12ab7f49366ef761951213e3a38f519a0a6b5d6534f1 · no open license confirmed · no open license confirmed
span-reshetar-first-time-meanparagraph beginning 'The numbers from 2019'em:dossier-span:sha256:6f0efeaf8fa67729f3f75793e1b64ce9961bb3463e5acc9d9c4a1d5aeb588520 · sha256:1452bea6f3158b2b58d90800d02a6cf2348935e6a194e07ab22efc88c5f26302em:dossier-edition:sha256:852a2ad23d9f6f760f0f21c4147eb6dd1b3c936ea9704e3dc3f743de193e4e49 · sha256:460f8d22d10526ee2a92e1fb4045e1cabacb0dc4a668e142a91a30592ccf0ed2em:dossier-source-work:sha256:802fad21dc2e69f5baff90d345e036b898bf958351af8eae39f346d23e005a12 · no open license confirmed · no open license confirmed

Calculation · analytic reproduction of the executable OSF inputs: {"derived_ube_sd":35.24729455256271,"sample_mbe_sd":17.74327194332649,"z_at_0_27":-0.6128129910166272}; equation=null; comparison_population=null; depends_on=[].

Relation: relation-assertion-derive-martinez-parameters em:dossier-evidence-relation:sha256:592ad3c88ad3d542736bb92ba9aeab1c3f375fd40d9688295eb6e8021a7d88e7
Proposition: em:dossier-proposition:sha256:8a265341b3cca87de499aa7fe9807fcb85405a680c9899fb2844a96717923ac6
Assertion: em:dossier-assertion:sha256:46b0fa65ec0a5d666de7e82ebfe7ea040c7bccf9d1862d67389fdc9ae6f6d6cb

{
  "derivation": {
    "derivation_id": "derive-martinez-parameters",
    "input_cell_ids": [
      "cell-martinez-july-mbe-85",
      "cell-martinez-july-mbe-90",
      "cell-martinez-july-mbe-95",
      "cell-martinez-july-mbe-100",
      "cell-martinez-july-mbe-105",
      "cell-martinez-july-mbe-110",
      "cell-martinez-july-mbe-115",
      "cell-martinez-july-mbe-120",
      "cell-martinez-july-mbe-125",
      "cell-martinez-july-mbe-130",
      "cell-martinez-july-mbe-135",
      "cell-martinez-july-mbe-140",
      "cell-martinez-july-mbe-145",
      "cell-martinez-july-mbe-150",
      "cell-martinez-july-mbe-155",
      "cell-martinez-july-mbe-160",
      "cell-martinez-july-mbe-165",
      "cell-martinez-july-mbe-170",
      "cell-martinez-july-mbe-175",
      "cell-martinez-july-mbe-180",
      "cell-martinez-july-mbe-185",
      "cell-ncbe-ube-score-266",
      "cell-ny-first-timers-rate"
    ],
    "input_span_ids": [
      "span-reshetar-first-time-mean",
      "span-martinez-mean-assumption",
      "span-martinez-script-july-mbe-distribution",
      "span-martinez-script-ube-sd",
      "span-ncbe-ny-cutoff-2022",
      "span-ny-first-timers-2022"
    ],
    "inputs": {
      "assumed_first_time_essay_mean": 143.8,
      "assumed_first_time_ube_mean": 287.6,
      "first_time_mbe_mean": 143.8,
      "july_mbe_binned_observations": 1002,
      "new_york_cutoff": 266.0,
      "new_york_nonpass_proportion": 0.27
    },
    "method": "analytic reproduction of the executable OSF inputs",
    "results": {
      "derived_ube_sd": 35.24729455256271,
      "sample_mbe_sd": 17.74327194332649,
      "z_at_0_27": -0.6128129910166272
    },
    "uncertainty": "The UBE distribution is inferred from aggregate inputs and normality; the essay mean/SD are assumed rather than observed."
  },
  "resolved_input_cells": [
    {
      "cell_id": "cell-martinez-july-mbe-85",
      "count": 2,
      "score": 85
    },
    {
      "cell_id": "cell-martinez-july-mbe-90",
      "count": 2,
      "score": 90
    },
    {
      "cell_id": "cell-martinez-july-mbe-95",
      "count": 5,
      "score": 95
    },
    {
      "cell_id": "cell-martinez-july-mbe-100",
      "count": 6,
      "score": 100
    },
    {
      "cell_id": "cell-martinez-july-mbe-105",
      "count": 13,
      "score": 105
    },
    {
      "cell_id": "cell-martinez-july-mbe-110",
      "count": 22,
      "score": 110
    },
    {
      "cell_id": "cell-martinez-july-mbe-115",
      "count": 33,
      "score": 115
    },
    {
      "cell_id": "cell-martinez-july-mbe-120",
      "count": 56,
      "score": 120
    },
    {
      "cell_id": "cell-martinez-july-mbe-125",
      "count": 73,
      "score": 125
    },
    {
      "cell_id": "cell-martinez-july-mbe-130",
      "count": 78,
      "score": 130
    },
    {
      "cell_id": "cell-martinez-july-mbe-135",
      "count": 104,
      "score": 135
    },
    {
      "cell_id": "cell-martinez-july-mbe-140",
      "count": 96,
      "score": 140
    },
    {
      "cell_id": "cell-martinez-july-mbe-145",
      "count": 101,
      "score": 145
    },
    {
      "cell_id": "cell-martinez-july-mbe-150",
      "count": 99,
      "score": 150
    },
    {
      "cell_id": "cell-martinez-july-mbe-155",
      "count": 99,
      "score": 155
    },
    {
      "cell_id": "cell-martinez-july-mbe-160",
      "count": 79,
      "score": 160
    },
    {
      "cell_id": "cell-martinez-july-mbe-165",
      "count": 64,
      "score": 165
    },
    {
      "cell_id": "cell-martinez-july-mbe-170",
      "count": 38,
      "score": 170
    },
    {
      "cell_id": "cell-martinez-july-mbe-175",
      "count": 22,
      "score": 175
    },
    {
      "cell_id": "cell-martinez-july-mbe-180",
      "count": 8,
      "score": 180
    },
    {
      "cell_id": "cell-martinez-july-mbe-185",
      "count": 2,
      "score": 185
    },
    {
      "cell_id": "cell-ncbe-ube-score-266",
      "column": "Score",
      "row": "266",
      "text": "266"
    },
    {
      "cell_id": "cell-ny-first-timers-rate",
      "column": "Rate",
      "row": "ALL Candidates",
      "text": "73%"
    }
  ]
}

span-calculation-derive-martinez-parametersaccepted derivation and input cells: derive-martinez-parametersem:dossier-span:sha256:b36533d6f13983fa968a987d5263ea1ab6220a87e2b416ab59a0c3d40ea03d09 · sha256:b92c0e5f9e2866c3806a45f478310c538513bb17644f78c91900edf5fc547d2eem:dossier-edition:sha256:32d3146687af86ae7865d08fbc1636a0c0aa6195e52cb00350ce4b0d525ade2c · sha256:ab330f3d40dfdb5e467da1f66c8f2d318b6b5ed88b770555956d3c5c79151a4bem:dossier-source-work:sha256:bb86be7b07b85a8d09a4fab4b2526a49c82b959ee3fb31bcf33ff7a5e6e27831 · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions
span-martinez-mean-assumption§3.1, journal p. 588em:dossier-span:sha256:ecc711638dfc4b30115783c7b8d7f32b292b8c723a36f8dfbf421e41923b9461 · sha256:d292fbd6a0a2c16559d69c651b273374cef4c7383b7c2605b97f85381bf8c62dem:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667 · sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-july-mbe-distributionlines 55-58em:dossier-span:sha256:c93f04ab5c92ae7955b75278cff1f0392210fc5462823dbe30cba42b5ffbab38 · sha256:9912e2d4f1e369b4f9529d27b6d1fef5936814f0c6151a9fe9f675af8c375e0cem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-ube-sdlines 104-112em:dossier-span:sha256:7c297fee53ebcae4ddc2b4bd2301d9742b570914c364b3993890c92c3ab42d83 · sha256:29d389334aa2b12045b1d9febb630b66d80d832615d0764534e4e0a2ac49b55bem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-ncbe-ny-cutoff-2022Minimum Passing UBE Score by Jurisdiction in 2022, row 266em:dossier-span:sha256:160241d336f3cc06be5b4d52f6ac9229563916a9ca197dbbacb968b7c60981f8 · sha256:8e029f660745d0cb0de2acd77e8d6e8c8f9c544d1ba560c3250bc30861ad653bem:dossier-edition:sha256:2000492e8e5040cd8f5eaa216e7fc1d4d21d4f58afbfc6af5a218f4f53333081 · sha256:b55af410679b66b8a0057d5f8dd17baec7985675c94d7af44818f5c4c418cfb7em:dossier-source-work:sha256:324982bb2b90295b84f9e309ab84ca9a46cd0f1be9e9e3079a18bcc20bf58e63 · no open license confirmed · no open license confirmed
span-ny-first-timers-2022FIRST TIMERS, ALL Candidates, Combined 2022em:dossier-span:sha256:468763dc341321dd638ef9a08f6606ac5e02806a4086e4b0ba7bc05400c1d703 · sha256:e55680bc3d09d5c2968a8449c122db232a89c5f89b29dc9944b69c2b1eeeee11em:dossier-edition:sha256:10bbf9ff899ba3ad12de154e0ab9ea7cc3f34c0f2263e79951e26539911bec5f · sha256:bbdcb28fb38b0d7d03ddb4e4bcfbcca170d2443491a180fee0978c7510cdb74fem:dossier-source-work:sha256:51aa3262776f5da4a8ef12ab7f49366ef761951213e3a38f519a0a6b5d6534f1 · no open license confirmed · no open license confirmed
span-reshetar-first-time-meanparagraph beginning 'The numbers from 2019'em:dossier-span:sha256:6f0efeaf8fa67729f3f75793e1b64ce9961bb3463e5acc9d9c4a1d5aeb588520 · sha256:1452bea6f3158b2b58d90800d02a6cf2348935e6a194e07ab22efc88c5f26302em:dossier-edition:sha256:852a2ad23d9f6f760f0f21c4147eb6dd1b3c936ea9704e3dc3f743de193e4e49 · sha256:460f8d22d10526ee2a92e1fb4045e1cabacb0dc4a668e142a91a30592ccf0ed2em:dossier-source-work:sha256:802fad21dc2e69f5baff90d345e036b898bf958351af8eae39f346d23e005a12 · no open license confirmed · no open license confirmed

Calculation · normal CDF conditional on modeled essay score >= 135: 15.252536216882072; equation=null; comparison_population="modeled essay scores at or above 135"; depends_on=["derive-martinez-parameters"].

Relation: relation-assertion-derive-martinez-passers-essay em:dossier-evidence-relation:sha256:ab1d937208a7dd4488c51004af267a03414306012ee327ecf10c05d0645a32b7
Proposition: em:dossier-proposition:sha256:6c97aaa7a9bc70f03c07f91df685245b1a1278b97bb4e9bcf56f7e85861dda76
Assertion: em:dossier-assertion:sha256:bfe2fd3438fb2f351efa67ef5066876074b428fd1fcd8b6c379070873fda676b

{
  "derivation": {
    "comparison_population": "modeled essay scores at or above 135",
    "depends_on": [
      "derive-martinez-parameters"
    ],
    "derivation_id": "derive-martinez-passers-essay",
    "method": "normal CDF conditional on modeled essay score >= 135",
    "result_percentile": 15.252536216882072
  },
  "resolved_input_cells": []
}

span-calculation-derive-martinez-passers-essayaccepted derivation and input cells: derive-martinez-passers-essayem:dossier-span:sha256:75c94b1ed032d669e53b051fa98d8234be91d8062a23ee43147ac17553d9e5db · sha256:e28cf7118b36578f51acca1366d26808d9189b9b699ff2f8ce7c4b77bde9b468em:dossier-edition:sha256:32d3146687af86ae7865d08fbc1636a0c0aa6195e52cb00350ce4b0d525ade2c · sha256:ab330f3d40dfdb5e467da1f66c8f2d318b6b5ed88b770555956d3c5c79151a4bem:dossier-source-work:sha256:bb86be7b07b85a8d09a4fab4b2526a49c82b959ee3fb31bcf33ff7a5e6e27831 · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions
span-martinez-mean-assumption§3.1, journal p. 588em:dossier-span:sha256:ecc711638dfc4b30115783c7b8d7f32b292b8c723a36f8dfbf421e41923b9461 · sha256:d292fbd6a0a2c16559d69c651b273374cef4c7383b7c2605b97f85381bf8c62dem:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667 · sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-july-mbe-distributionlines 55-58em:dossier-span:sha256:c93f04ab5c92ae7955b75278cff1f0392210fc5462823dbe30cba42b5ffbab38 · sha256:9912e2d4f1e369b4f9529d27b6d1fef5936814f0c6151a9fe9f675af8c375e0cem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-ube-sdlines 104-112em:dossier-span:sha256:7c297fee53ebcae4ddc2b4bd2301d9742b570914c364b3993890c92c3ab42d83 · sha256:29d389334aa2b12045b1d9febb630b66d80d832615d0764534e4e0a2ac49b55bem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-ncbe-ny-cutoff-2022Minimum Passing UBE Score by Jurisdiction in 2022, row 266em:dossier-span:sha256:160241d336f3cc06be5b4d52f6ac9229563916a9ca197dbbacb968b7c60981f8 · sha256:8e029f660745d0cb0de2acd77e8d6e8c8f9c544d1ba560c3250bc30861ad653bem:dossier-edition:sha256:2000492e8e5040cd8f5eaa216e7fc1d4d21d4f58afbfc6af5a218f4f53333081 · sha256:b55af410679b66b8a0057d5f8dd17baec7985675c94d7af44818f5c4c418cfb7em:dossier-source-work:sha256:324982bb2b90295b84f9e309ab84ca9a46cd0f1be9e9e3079a18bcc20bf58e63 · no open license confirmed · no open license confirmed
span-ny-first-timers-2022FIRST TIMERS, ALL Candidates, Combined 2022em:dossier-span:sha256:468763dc341321dd638ef9a08f6606ac5e02806a4086e4b0ba7bc05400c1d703 · sha256:e55680bc3d09d5c2968a8449c122db232a89c5f89b29dc9944b69c2b1eeeee11em:dossier-edition:sha256:10bbf9ff899ba3ad12de154e0ab9ea7cc3f34c0f2263e79951e26539911bec5f · sha256:bbdcb28fb38b0d7d03ddb4e4bcfbcca170d2443491a180fee0978c7510cdb74fem:dossier-source-work:sha256:51aa3262776f5da4a8ef12ab7f49366ef761951213e3a38f519a0a6b5d6534f1 · no open license confirmed · no open license confirmed
span-reshetar-first-time-meanparagraph beginning 'The numbers from 2019'em:dossier-span:sha256:6f0efeaf8fa67729f3f75793e1b64ce9961bb3463e5acc9d9c4a1d5aeb588520 · sha256:1452bea6f3158b2b58d90800d02a6cf2348935e6a194e07ab22efc88c5f26302em:dossier-edition:sha256:852a2ad23d9f6f760f0f21c4147eb6dd1b3c936ea9704e3dc3f743de193e4e49 · sha256:460f8d22d10526ee2a92e1fb4045e1cabacb0dc4a668e142a91a30592ccf0ed2em:dossier-source-work:sha256:802fad21dc2e69f5baff90d345e036b898bf958351af8eae39f346d23e005a12 · no open license confirmed · no open license confirmed

Calculation · normal CDF conditional on modeled MBE score >= 135: 69.31081545343784; equation=null; comparison_population="modeled MBE scores at or above 135"; depends_on=["derive-martinez-parameters"].

Relation: relation-assertion-derive-martinez-passers-mbe em:dossier-evidence-relation:sha256:1dd0e17278567855ddd02c50c7a61cc75f28fc83737928dcc68e610cce4ba6b2
Proposition: em:dossier-proposition:sha256:c564ce54eb3b2d6f72781fee0e99f5162aec79993a6aa87f999d7b8d2775420e
Assertion: em:dossier-assertion:sha256:837a7e443196825e489e5e7f7a09e74cdcd36ce65a43cf63c504a479c59da0a5

{
  "derivation": {
    "comparison_population": "modeled MBE scores at or above 135",
    "depends_on": [
      "derive-martinez-parameters"
    ],
    "derivation_id": "derive-martinez-passers-mbe",
    "method": "normal CDF conditional on modeled MBE score >= 135",
    "result_percentile": 69.31081545343784
  },
  "resolved_input_cells": []
}

span-calculation-derive-martinez-passers-mbeaccepted derivation and input cells: derive-martinez-passers-mbeem:dossier-span:sha256:25dbcaa16173c2875dcab51c6fbd9210df50cf2aa03cb3e2514724edcc667200 · sha256:bc34a7eda1879f438d7e7945440d3db419371f000787488a1830b7c283f81845em:dossier-edition:sha256:32d3146687af86ae7865d08fbc1636a0c0aa6195e52cb00350ce4b0d525ade2c · sha256:ab330f3d40dfdb5e467da1f66c8f2d318b6b5ed88b770555956d3c5c79151a4bem:dossier-source-work:sha256:bb86be7b07b85a8d09a4fab4b2526a49c82b959ee3fb31bcf33ff7a5e6e27831 · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions
span-martinez-mean-assumption§3.1, journal p. 588em:dossier-span:sha256:ecc711638dfc4b30115783c7b8d7f32b292b8c723a36f8dfbf421e41923b9461 · sha256:d292fbd6a0a2c16559d69c651b273374cef4c7383b7c2605b97f85381bf8c62dem:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667 · sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-july-mbe-distributionlines 55-58em:dossier-span:sha256:c93f04ab5c92ae7955b75278cff1f0392210fc5462823dbe30cba42b5ffbab38 · sha256:9912e2d4f1e369b4f9529d27b6d1fef5936814f0c6151a9fe9f675af8c375e0cem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-ube-sdlines 104-112em:dossier-span:sha256:7c297fee53ebcae4ddc2b4bd2301d9742b570914c364b3993890c92c3ab42d83 · sha256:29d389334aa2b12045b1d9febb630b66d80d832615d0764534e4e0a2ac49b55bem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-ncbe-ny-cutoff-2022Minimum Passing UBE Score by Jurisdiction in 2022, row 266em:dossier-span:sha256:160241d336f3cc06be5b4d52f6ac9229563916a9ca197dbbacb968b7c60981f8 · sha256:8e029f660745d0cb0de2acd77e8d6e8c8f9c544d1ba560c3250bc30861ad653bem:dossier-edition:sha256:2000492e8e5040cd8f5eaa216e7fc1d4d21d4f58afbfc6af5a218f4f53333081 · sha256:b55af410679b66b8a0057d5f8dd17baec7985675c94d7af44818f5c4c418cfb7em:dossier-source-work:sha256:324982bb2b90295b84f9e309ab84ca9a46cd0f1be9e9e3079a18bcc20bf58e63 · no open license confirmed · no open license confirmed
span-ny-first-timers-2022FIRST TIMERS, ALL Candidates, Combined 2022em:dossier-span:sha256:468763dc341321dd638ef9a08f6606ac5e02806a4086e4b0ba7bc05400c1d703 · sha256:e55680bc3d09d5c2968a8449c122db232a89c5f89b29dc9944b69c2b1eeeee11em:dossier-edition:sha256:10bbf9ff899ba3ad12de154e0ab9ea7cc3f34c0f2263e79951e26539911bec5f · sha256:bbdcb28fb38b0d7d03ddb4e4bcfbcca170d2443491a180fee0978c7510cdb74fem:dossier-source-work:sha256:51aa3262776f5da4a8ef12ab7f49366ef761951213e3a38f519a0a6b5d6534f1 · no open license confirmed · no open license confirmed
span-reshetar-first-time-meanparagraph beginning 'The numbers from 2019'em:dossier-span:sha256:6f0efeaf8fa67729f3f75793e1b64ce9961bb3463e5acc9d9c4a1d5aeb588520 · sha256:1452bea6f3158b2b58d90800d02a6cf2348935e6a194e07ab22efc88c5f26302em:dossier-edition:sha256:852a2ad23d9f6f760f0f21c4147eb6dd1b3c936ea9704e3dc3f743de193e4e49 · sha256:460f8d22d10526ee2a92e1fb4045e1cabacb0dc4a668e142a91a30592ccf0ed2em:dossier-source-work:sha256:802fad21dc2e69f5baff90d345e036b898bf958351af8eae39f346d23e005a12 · no open license confirmed · no open license confirmed

Calculation · normal CDF conditional on modeled UBE score >= 270: 44.45020553281165; equation="100 * (F(298) - F(270)) / (1 - F(270))"; comparison_population="modeled first-time scores at or above 270"; depends_on=["derive-martinez-parameters"].

Relation: relation-assertion-derive-martinez-passers-ube em:dossier-evidence-relation:sha256:20f088f009065cfc3380add119f1dd245d0cc60c26ce7304d2603d810dcceac5
Proposition: em:dossier-proposition:sha256:9119f9ff62ea1e7760aabb53df19e6824e081aaab2d28ddac873b49ba1829c69
Assertion: em:dossier-assertion:sha256:0e84c467e4cdeee807106e9fa021760a1a2e63bcd7fb41c4cfccea488558b23f

{
  "derivation": {
    "comparison_population": "modeled first-time scores at or above 270",
    "depends_on": [
      "derive-martinez-parameters"
    ],
    "derivation_id": "derive-martinez-passers-ube",
    "equation": "100 * (F(298) - F(270)) / (1 - F(270))",
    "method": "normal CDF conditional on modeled UBE score >= 270",
    "result_percentile": 44.45020553281165,
    "uncertainty": "The script uses 270 for this filter after using New York's 266 cutoff to infer UBE SD."
  },
  "resolved_input_cells": []
}

span-calculation-derive-martinez-passers-ubeaccepted derivation and input cells: derive-martinez-passers-ubeem:dossier-span:sha256:3738518581952127b4be8dbb224e03a6eb3126a5ec5c685a837cba9e0d7bfcde · sha256:5cd325ee19d0d60ca21ed01dcf1cc3c6cfb1edfb2f9fb702d680f03a2ec1a8f3em:dossier-edition:sha256:32d3146687af86ae7865d08fbc1636a0c0aa6195e52cb00350ce4b0d525ade2c · sha256:ab330f3d40dfdb5e467da1f66c8f2d318b6b5ed88b770555956d3c5c79151a4bem:dossier-source-work:sha256:bb86be7b07b85a8d09a4fab4b2526a49c82b959ee3fb31bcf33ff7a5e6e27831 · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions
span-martinez-mean-assumption§3.1, journal p. 588em:dossier-span:sha256:ecc711638dfc4b30115783c7b8d7f32b292b8c723a36f8dfbf421e41923b9461 · sha256:d292fbd6a0a2c16559d69c651b273374cef4c7383b7c2605b97f85381bf8c62dem:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667 · sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-july-mbe-distributionlines 55-58em:dossier-span:sha256:c93f04ab5c92ae7955b75278cff1f0392210fc5462823dbe30cba42b5ffbab38 · sha256:9912e2d4f1e369b4f9529d27b6d1fef5936814f0c6151a9fe9f675af8c375e0cem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-ube-sdlines 104-112em:dossier-span:sha256:7c297fee53ebcae4ddc2b4bd2301d9742b570914c364b3993890c92c3ab42d83 · sha256:29d389334aa2b12045b1d9febb630b66d80d832615d0764534e4e0a2ac49b55bem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-ncbe-ny-cutoff-2022Minimum Passing UBE Score by Jurisdiction in 2022, row 266em:dossier-span:sha256:160241d336f3cc06be5b4d52f6ac9229563916a9ca197dbbacb968b7c60981f8 · sha256:8e029f660745d0cb0de2acd77e8d6e8c8f9c544d1ba560c3250bc30861ad653bem:dossier-edition:sha256:2000492e8e5040cd8f5eaa216e7fc1d4d21d4f58afbfc6af5a218f4f53333081 · sha256:b55af410679b66b8a0057d5f8dd17baec7985675c94d7af44818f5c4c418cfb7em:dossier-source-work:sha256:324982bb2b90295b84f9e309ab84ca9a46cd0f1be9e9e3079a18bcc20bf58e63 · no open license confirmed · no open license confirmed
span-ny-first-timers-2022FIRST TIMERS, ALL Candidates, Combined 2022em:dossier-span:sha256:468763dc341321dd638ef9a08f6606ac5e02806a4086e4b0ba7bc05400c1d703 · sha256:e55680bc3d09d5c2968a8449c122db232a89c5f89b29dc9944b69c2b1eeeee11em:dossier-edition:sha256:10bbf9ff899ba3ad12de154e0ab9ea7cc3f34c0f2263e79951e26539911bec5f · sha256:bbdcb28fb38b0d7d03ddb4e4bcfbcca170d2443491a180fee0978c7510cdb74fem:dossier-source-work:sha256:51aa3262776f5da4a8ef12ab7f49366ef761951213e3a38f519a0a6b5d6534f1 · no open license confirmed · no open license confirmed
span-reshetar-first-time-meanparagraph beginning 'The numbers from 2019'em:dossier-span:sha256:6f0efeaf8fa67729f3f75793e1b64ce9961bb3463e5acc9d9c4a1d5aeb588520 · sha256:1452bea6f3158b2b58d90800d02a6cf2348935e6a194e07ab22efc88c5f26302em:dossier-edition:sha256:852a2ad23d9f6f760f0f21c4147eb6dd1b3c936ea9704e3dc3f743de193e4e49 · sha256:460f8d22d10526ee2a92e1fb4045e1cabacb0dc4a668e142a91a30592ccf0ed2em:dossier-source-work:sha256:802fad21dc2e69f5baff90d345e036b898bf958351af8eae39f346d23e005a12 · no open license confirmed · no open license confirmed

percentile_analysis_new.Rmd

§3.1, journal p. 588

{ "format": "exact-segments", "locator": "\u00a73.1, journal p. 588", "normalization": "collapse-whitespace", "segments": [ { "segment_id": "segment-martinez-essay-mean", "text": "Thus, the methodology here assumed that the mean first-time essay score is 143.8." }, { "segment_id": "segment-martinez-ube-mean", "text": "Given that the total UBE score is computed directly by adding MBE and essay scores (National Conference of Bar Examiners n.d.-h), an assumption was made that mean first-time UBE score is 287.6 (143.8 + 143.8)." } ], "span_id": "span-martinez-mean-assumption", "supports": "The UBE mean is derived from an assumed essay mean, not directly observed as a national UBE mean." }

Work: em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b
Edition: em:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667
Edition digest: sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5
Span: em:dossier-span:sha256:ecc711638dfc4b30115783c7b8d7f32b292b8c723a36f8dfbf421e41923b9461
Span digest: sha256:d292fbd6a0a2c16559d69c651b273374cef4c7383b7c2605b97f85381bf8c62d
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed

percentile_analysis_new.Rmd

§3.1, journal pp. 588-589

Assuming that UBE scores (as well as MBE and essay subscores) are normally distributed, percentiles of GPT’s score can be directly computed after computing the parameters of these distributions (i.e. the mean and standard deviation).

Work: em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b
Edition: em:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667
Edition digest: sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5
Span: em:dossier-span:sha256:cceeb7bbb8f7496871eeb81724f8be4bcd784bde0cbf28a236ecd81d5ad7b8b3
Span digest: sha256:3a80dd544d7516c8d33d8b55a4cdf44a2975a279ad377d53b09927cfcc26a733
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed

percentile_analysis_new.Rmd

lines 104-112

{ "format": "code-segment-transcription", "locator": "lines 104-112", "normalization": "exact-code-text", "segments": [ { "segment_id": "segment-martinez-code-mean", "text": "mean <- 287.6" }, { "segment_id": "segment-martinez-code-quantile", "text": "quantile_value <- 266" }, { "segment_id": "segment-martinez-code-percentile", "text": "percentile <- 0.27 # 24%" }, { "segment_id": "segment-martinez-code-z-score", "text": "z_score <- qnorm(percentile)" }, { "segment_id": "segment-martinez-code-ube-sd", "text": "sd_UBE <- (quantile_value - mean) / z_score" } ], "span_id": "span-martinez-script-ube-sd", "supports": "The script derives UBE standard deviation from mean 287.6, cutoff 266, and non-pass proportion 0.27." }

Work: em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b
Edition: em:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745
Edition digest: sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666
Span: em:dossier-span:sha256:7c297fee53ebcae4ddc2b4bd2301d9742b570914c364b3993890c92c3ab42d83
Span digest: sha256:29d389334aa2b12045b1d9febb630b66d80d832615d0764534e4e0a2ac49b55b
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed

First-Time Exam Takers and Repeaters in 2022

Minimum Passing UBE Score by Jurisdiction in 2022, row 266

{ "cells": [ { "cell_id": "cell-ncbe-ube-score-266", "column": "Score", "row": "266", "text": "266" }, { "cell_id": "cell-ncbe-ube-jurisdictions-266", "column": "Jurisdictions", "row": "266", "text": "Connecticut; District of Columbia; Illinois; Iowa; Kansas; Kentucky; Maryland; Montana; New Jersey; New York; South Carolina; Virgin Islands" } ], "format": "table-cell-transcription", "locator": "Minimum Passing UBE Score by Jurisdiction in 2022, row 266", "normalization": "exact-cell-text", "span_id": "span-ncbe-ny-cutoff-2022", "supports": "New York's 2022 minimum passing UBE score was 266." }

Work: em:dossier-source-work:sha256:324982bb2b90295b84f9e309ab84ca9a46cd0f1be9e9e3079a18bcc20bf58e63
Edition: em:dossier-edition:sha256:2000492e8e5040cd8f5eaa216e7fc1d4d21d4f58afbfc6af5a218f4f53333081
Edition digest: sha256:b55af410679b66b8a0057d5f8dd17baec7985675c94d7af44818f5c4c418cfb7
Span: em:dossier-span:sha256:160241d336f3cc06be5b4d52f6ac9229563916a9ca197dbbacb968b7c60981f8
Span digest: sha256:8e029f660745d0cb0de2acd77e8d6e8c8f9c544d1ba560c3250bc30861ad653b
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

New York Bar Exam 2022 Statistics

FIRST TIMERS, ALL Candidates, Combined 2022

{ "cells": [ { "cell_id": "cell-ny-first-timers-took", "column": "Took", "row": "ALL Candidates", "text": "9,457" }, { "cell_id": "cell-ny-first-timers-passed", "column": "Passed", "row": "ALL Candidates", "text": "6,867" }, { "cell_id": "cell-ny-first-timers-rate", "column": "Rate", "row": "ALL Candidates", "text": "73%" } ], "format": "table-cell-transcription", "locator": "FIRST TIMERS, ALL Candidates, Combined 2022", "normalization": "exact-cell-text", "span_id": "span-ny-first-timers-2022", "supports": "The combined first-time pass rate was 73%, so the complementary non-pass proportion used by the re-analysis is 27%." }

Work: em:dossier-source-work:sha256:51aa3262776f5da4a8ef12ab7f49366ef761951213e3a38f519a0a6b5d6534f1
Edition: em:dossier-edition:sha256:10bbf9ff899ba3ad12de154e0ab9ea7cc3f34c0f2263e79951e26539911bec5f
Edition digest: sha256:bbdcb28fb38b0d7d03ddb4e4bcfbcca170d2443491a180fee0978c7510cdb74f
Span: em:dossier-span:sha256:468763dc341321dd638ef9a08f6606ac5e02806a4086e4b0ba7bc05400c1d703
Span digest: sha256:e55680bc3d09d5c2968a8449c122db232a89c5f89b29dc9944b69c2b1eeeee11
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

Illinois February 2018 Bar Examination Percentile Equivalents

total-scale table, rows 300 and 290

{ "cells": [ { "cell_id": "cell-illinois-feb-2018-300", "percentile": 90, "score": 300 }, { "cell_id": "cell-illinois-feb-2018-290", "percentile": 85, "score": 290 } ], "format": "table-cell-transcription", "locator": "total-scale table, rows 300 and 290", "normalization": "exact-cell-text", "span_id": "span-illinois-feb-2018-anchors", "supports": "The chart brackets 298 but contains no printed 298 row or interpolation rule." }

Work: em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd
Edition: em:dossier-edition:sha256:9be0e724fc35f74c96e635c6263f8f44ef804c3406c34b268bb710b24228a368
Edition digest: sha256:44de7bd89cd4aeacee7b38d7077ff659e3baf552cb9b295401ec9b7229c04964
Span: em:dossier-span:sha256:56a011a09e4e7dd05afbc0309c94d4dcfa1ecf1d6fd3df6a6a6f65204c7ccf03
Span digest: sha256:b3b8f789db7571a4fd8ed73e5380f2847aae920172722b9f6f51eec96c62e9ce
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

Illinois February 2018 Bar Examination Percentile Equivalents

total-scale table, rows 300 and 290

{ "cells": [ { "cell_id": "cell-illinois-feb-2019-300", "percentile": 90, "score": 300 }, { "cell_id": "cell-illinois-feb-2019-290", "percentile": 83, "score": 290 } ], "format": "table-cell-transcription", "locator": "total-scale table, rows 300 and 290", "normalization": "exact-cell-text", "span_id": "span-illinois-feb-2019-anchors", "supports": "This later February chart also brackets 298 but supplies no interpolation rule." }

Work: em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd
Edition: em:dossier-edition:sha256:2c2badc4c75c58ca49007bda1f0c1fcad4b301ecd060bc905bd8a741acd80925
Edition digest: sha256:91185bd6263d73e5976b5b00f28fb9b62301be0bb0347b1d0a308c295f6a6151
Span: em:dossier-span:sha256:1e2736835309f364d7f8553b6e26a7c9802dce86c720c84fa2097f7e3a755ca3
Span digest: sha256:29f7ab38f7ff54e4576aa5b3c6fd310debd0520513f68fc3e21744af073923f8
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

Illinois February 2018 Bar Examination Percentile Equivalents

total-scale table, rows 300 and 290

{ "cells": [ { "cell_id": "cell-illinois-jul-2018-300", "percentile": 70, "score": 300 }, { "cell_id": "cell-illinois-jul-2018-290", "percentile": 59, "score": 290 } ], "format": "table-cell-transcription", "locator": "total-scale table, rows 300 and 290", "normalization": "exact-cell-text", "span_id": "span-illinois-jul-2018-anchors", "supports": "The July chart places the same score region far below the February chart." }

Work: em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd
Edition: em:dossier-edition:sha256:2b720aa3b7368249b079f4bc7d3de383efc590510bc33e1ed88db133e9293237
Edition digest: sha256:a9aa31983924ffec8d0e713ad29fc16a0e466557a57eba5061577afc4698bae6
Span: em:dossier-span:sha256:3ec4b97842a7b34ea0a2857a50da7f6a02eb1556351c7ec727b2cccd2743637c
Span digest: sha256:fe399dcc350821ac8ab546593931f1e34b24d811cad9062126e487a53783a783
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

Appendix for GPT-4 passes the Bar Exam

footnote 5

While we are not fully convinced of the methodological approach taken in some subsequent analysis [78], we do agree that it would be better to consider the raw 297 UBE as falling within a range between 68th and 90th percentile (depending on the precise state and timing of the exam administration).

Work: em:dossier-source-work:sha256:880443d56eafdb224149d953f7de48e7550d53f814008551895b180ecb4b203b
Edition: em:dossier-edition:sha256:21621ede6a23bb3f98867c0918e5d9808d6dced5ca3f271a3a52e561810f2f7b
Edition digest: sha256:9d10cfedd531e62d96ea27bb78b00a9148e6b5d8135807836c5982e65a42416e
Span: em:dossier-span:sha256:477c595f89b6efc4f4fac04d627e65d95daa7929d10f76d3486a96c6c8bf8b06
Span digest: sha256:7e376175b36f8e2809bc74e378c7354858289a689370b70f82df2965d32b9026
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree · CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree

First-Time Exam Takers and Repeaters in 2022

table note

NOTE: First-time exam and repeat test taker data supplied by the jurisdictions in this chart are based on those examinees’ testing experience in the reporting jurisdiction only and do not account for possible previous attempts at the bar examination in other jurisdictions.

Work: em:dossier-source-work:sha256:324982bb2b90295b84f9e309ab84ca9a46cd0f1be9e9e3079a18bcc20bf58e63
Edition: em:dossier-edition:sha256:b3dddcd76a7bbe208e296d4b9a13fc53e91cd14df2ae42ee1c91803569240eca
Edition digest: sha256:f42357f4f20e9db92a1a48755cb2f269123cb4ee93cecc042bff83d990989794
Span: em:dossier-span:sha256:f8fe10f696eb9dc37fa067a0b10401f5bf1e29589d6c6071cfaa44c15e59b183
Span digest: sha256:1feddb6e088dae626d7d1cc8f6a0207c19ac7e9f736830bf3a3ae4085e374a42
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

First-Time Exam Takers and Repeaters in 2022

NCBE MBE-based data, February and July 2022 totals

{ "format": "exact-segments", "locator": "NCBE MBE-based data, February and July 2022 totals", "normalization": "collapse-whitespace", "segments": [ { "segment_id": "segment-ncbe-february-repeaters-count", "text": "February likely repeaters taking: 11,289" }, { "segment_id": "segment-ncbe-february-repeaters-percent", "text": "68% of February 2022 examinees were likely repeaters" }, { "segment_id": "segment-ncbe-february-first-timers-count", "text": "February likely first-timers taking: 5,215" }, { "segment_id": "segment-ncbe-february-first-timers-percent", "text": "32% of February 2022 examinees were likely first-time takers" }, { "segment_id": "segment-ncbe-july-repeaters-count", "text": "July likely repeaters taking: 10,200" }, { "segment_id": "segment-ncbe-july-repeaters-percent", "text": "23% of July 2022 examinees were likely repeaters" }, { "segment_id": "segment-ncbe-july-first-timers-count", "text": "July likely first-timers taking: 34,505" }, { "segment_id": "segment-ncbe-july-first-timers-percent", "text": "77% of July 2021 examinees were likely first-time takers" } ], "span_id": "span-ncbe-snapshot-composition", "supports": "February and July have materially different inferred first-time/repeater composition." }

Work: em:dossier-source-work:sha256:324982bb2b90295b84f9e309ab84ca9a46cd0f1be9e9e3079a18bcc20bf58e63
Edition: em:dossier-edition:sha256:5e5c17ba424f84711ba55a14e9bf2b2846c2c05389280d7e21e1f1b2ad37810d
Edition digest: sha256:d2ce119a524286fa88bbaf4438c9052d77a5b728990db41cbc8cad70dc6740c4
Span: em:dossier-span:sha256:352571f2e61d157a8943ca6216ef23f38b6ae5cde1b55838d1a424bcc4ee9f09
Span digest: sha256:c389829939a67dd8c1d353a75cb1bdbc6823160e0582ba17bf83ba6d1e939dbc
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

Appendix for GPT-4 passes the Bar Exam

§4(d), Table 7

{ "cells": [ { "cell_id": "cell-katz-gpt4-mbe", "column": "GPT-4", "row": "MBE", "text": "157 points" }, { "cell_id": "cell-katz-gpt4-mee", "column": "GPT-4", "row": "MEE", "text": "84 points" }, { "cell_id": "cell-katz-gpt4-mpt", "column": "GPT-4", "row": "MPT", "text": "56 points" }, { "cell_id": "cell-katz-gpt4-overall", "column": "GPT-4", "row": "overall score", "text": "297 points" } ], "format": "table-cell-transcription", "locator": "\u00a74(d), Table 7", "normalization": "exact-cell-text", "span_id": "span-katz-vor-components", "supports": "The version-of-record component scores sum to 297." }

Work: em:dossier-source-work:sha256:880443d56eafdb224149d953f7de48e7550d53f814008551895b180ecb4b203b
Edition: em:dossier-edition:sha256:21621ede6a23bb3f98867c0918e5d9808d6dced5ca3f271a3a52e561810f2f7b
Edition digest: sha256:9d10cfedd531e62d96ea27bb78b00a9148e6b5d8135807836c5982e65a42416e
Span: em:dossier-span:sha256:293687df5cf4b74d839237414173205591aacf4850103e1621aece4f21fb763b
Span digest: sha256:e6c808d4b4dcd243210518497a0c8788d06ee91d8571c474c13514c2c8afe04b
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree · CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree

percentile_analysis_new.Rmd

introduction, journal p. 584

The paper successfully replicates the MBE score of 158, but highlights several methodological issues in the grading of the MPT + MEE components of the exam, which call into question the validity of the essay score (140).

Work: em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b
Edition: em:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667
Edition digest: sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5
Span: em:dossier-span:sha256:47ce609849ae1ff1224df62ce9b453707dd9b7a93cd4413822c84178eac7f853
Span digest: sha256:11db34f6066f861f338266e98c11852743b727734f62c1a773ae257bf219e609
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed

percentile_analysis_new.Rmd

lines 55-58

{ "cells": [ { "cell_id": "cell-martinez-july-mbe-85", "count": 2, "score": 85 }, { "cell_id": "cell-martinez-july-mbe-90", "count": 2, "score": 90 }, { "cell_id": "cell-martinez-july-mbe-95", "count": 5, "score": 95 }, { "cell_id": "cell-martinez-july-mbe-100", "count": 6, "score": 100 }, { "cell_id": "cell-martinez-july-mbe-105", "count": 13, "score": 105 }, { "cell_id": "cell-martinez-july-mbe-110", "count": 22, "score": 110 }, { "cell_id": "cell-martinez-july-mbe-115", "count": 33, "score": 115 }, { "cell_id": "cell-martinez-july-mbe-120", "count": 56, "score": 120 }, { "cell_id": "cell-martinez-july-mbe-125", "count": 73, "score": 125 }, { "cell_id": "cell-martinez-july-mbe-130", "count": 78, "score": 130 }, { "cell_id": "cell-martinez-july-mbe-135", "count": 104, "score": 135 }, { "cell_id": "cell-martinez-july-mbe-140", "count": 96, "score": 140 }, { "cell_id": "cell-martinez-july-mbe-145", "count": 101, "score": 145 }, { "cell_id": "cell-martinez-july-mbe-150", "count": 99, "score": 150 }, { "cell_id": "cell-martinez-july-mbe-155", "count": 99, "score": 155 }, { "cell_id": "cell-martinez-july-mbe-160", "count": 79, "score": 160 }, { "cell_id": "cell-martinez-july-mbe-165", "count": 64, "score": 165 }, { "cell_id": "cell-martinez-july-mbe-170", "count": 38, "score": 170 }, { "cell_id": "cell-martinez-july-mbe-175", "count": 22, "score": 175 }, { "cell_id": "cell-martinez-july-mbe-180", "count": 8, "score": 180 }, { "cell_id": "cell-martinez-july-mbe-185", "count": 2, "score": 185 } ], "code_lines": [ { "line_id": "line-martinez-july-distribution-55", "text": "July_distribution <- c(rep(85, 2), rep(90, 2), rep(95, 5), rep(100, 6), rep(105, 13)," }, { "line_id": "line-martinez-july-distribution-56", "text": " rep(110, 22), rep(115, 33), rep(120, 56), rep(125, 73), rep(130, 78)," }, { "line_id": "line-martinez-july-distribution-57", "text": " rep(135, 104), rep(140, 96), rep(145, 101), rep(150, 99), rep(155, 99)," }, { "line_id": "line-martinez-july-distribution-58", "text": " rep(160, 79), rep(165, 64), rep(170, 38), rep(175, 22), rep(180, 8), rep(185, 2))" } ], "format": "code-table-transcription", "locator": "lines 55-58", "normalization": "exact-code-text", "span_id": "span-martinez-script-july-mbe-distribution", "supports": "The executable analysis expands these 21 score-count cells to estimate the July MBE sample standard deviation." }

Work: em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b
Edition: em:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745
Edition digest: sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666
Span: em:dossier-span:sha256:c93f04ab5c92ae7955b75278cff1f0392210fc5462823dbe30cba42b5ffbab38
Span digest: sha256:9912e2d4f1e369b4f9529d27b6d1fef5936814f0c6151a9fe9f675af8c375e0c
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed

percentile_analysis_new.Rmd

lines 166-168

{ "format": "code-segment-transcription", "locator": "lines 166-168", "normalization": "exact-code-text", "segments": [ { "segment_id": "segment-martinez-code-filter-ube", "text": "filtered_data_ube <- data_ube[data_ube >= 270]" }, { "segment_id": "segment-martinez-code-filter-mbe", "text": "filtered_data_mbe <- data_mbe[data_mbe >= 135]" }, { "segment_id": "segment-martinez-code-filter-essay", "text": "filtered_data_essay <- data_essay[data_essay >= 135]" } ], "span_id": "span-martinez-script-thresholds", "supports": "The passers analysis filters UBE at 270 even though the SD input uses New York's 266 cutoff." }

Work: em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b
Edition: em:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745
Edition digest: sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666
Span: em:dossier-span:sha256:7da66b96a0ba3914388da9f878232040b78dc3be4727b77afe738797c9e53c8b
Span digest: sha256:e4f31613c4c910062cc0c84109333b81d866660c9565c354ac91f5fc1f76716f
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed

The Testing Column: Why Are February Bar Exam Pass Rates Lower than July Pass Rates?

paragraph beginning 'The numbers from 2019'

The numbers from 2019, the last prepandemic year, are typical: all likely first-time test takers earned an average MBE score of 143.8. Likely repeaters, however, earned an average MBE score of 132.4.

Work: em:dossier-source-work:sha256:802fad21dc2e69f5baff90d345e036b898bf958351af8eae39f346d23e005a12
Edition: em:dossier-edition:sha256:852a2ad23d9f6f760f0f21c4147eb6dd1b3c936ea9704e3dc3f743de193e4e49
Edition digest: sha256:460f8d22d10526ee2a92e1fb4045e1cabacb0dc4a668e142a91a30592ccf0ed2
Span: em:dossier-span:sha256:6f0efeaf8fa67729f3f75793e1b64ce9961bb3463e5acc9d9c4a1d5aeb588520
Span digest: sha256:1452bea6f3158b2b58d90800d02a6cf2348935e6a194e07ab22efc88c5f26302
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

06 The Martínez version of record and code support roughly 45th under the encoded passers model, while its abstract and discussion say roughly 48th; the packet preserves 45/48 as an internal discrepancy.

Typed relation: support

Relation: relation-assertion-claim-martinez-passers-conflict em:dossier-evidence-relation:sha256:98f71b562a1c5d46cd4a56ad684083756ac580d2ecdc0c1db9cb1b0438094252
Proposition: claim-martinez-passers-conflict em:dossier-proposition:sha256:4c6ac4b24cbd908f89d988c4bb1dbc008e9ff3b5bb4fffaa313145d1762465ce
Assertion: assertion-claim-martinez-passers-conflict em:dossier-assertion:sha256:35a975b45cb9ebe6fed26d97fde109f534f7ce69613026271c16226ebf3c1182
Lineage: lineage-martinez-analysis-root em:dossier-lineage:sha256:e6057f0dbfa37d0264e471e67a3941297a56467e67cad715d0eb37490d320cdd

Typed dependence

edge-benchmark-illinois-chartslineage-illinois-comparison-root → lineage-martinez-analysis-rootem:dossier-evidence-relation:sha256:3bbbc190e0662f0d0cf22c4bf02a5abe6f9e71e95061afd3dc94f535c83fb57aaccepted_edge_id=edge-benchmark-illinois-charts; accepted_dimension=benchmark; from=lineage-illinois-comparison-root; to=lineage-martinez-analysis-root; effects=It is comparison data, not an additional GPT-4 performance root. | Population drift affects rank without changing the model score. | It is a sensitivity input, not a replication of the model run.
edge-citation-katz-to-martinezlineage-model-performance-root → lineage-martinez-analysis-rootem:dossier-evidence-relation:sha256:a8c16efd90634c0a152f0fdd7a0a02f8d844004ea05edec24ec4651bf00764ccaccepted_edge_id=edge-citation-katz-to-martinez; accepted_dimension=citation; from=lineage-model-performance-root; to=lineage-martinez-analysis-root; effects=Citation links the analysis lineage to the same reported score.
edge-comparison-class-first-time-passers--1lineage-ncbe-comparison-root → lineage-martinez-analysis-rootem:dossier-evidence-relation:sha256:c26568911130747cbc372db3bcb7c6a64cecaa0134fb60ab0def98d7ceb621e4accepted_edge_id=edge-comparison-class-first-time-passers; accepted_dimension=comparison-class; from=lineage-ncbe-comparison-root,lineage-new-york-pass-rate-root; to=lineage-martinez-analysis-root; effects=Comparison populations are not interchangeable. | Administration mix can change percentile without new model evidence. | It parameterizes the comparison model rather than testing GPT-4. | The threshold defines a modeled comparator boundary.
edge-comparison-class-first-time-passers--2lineage-new-york-pass-rate-root → lineage-martinez-analysis-rootem:dossier-evidence-relation:sha256:0b7e8f83a90479b952cdbf35c12046e8810bf312a6f413158f9d4567bc45849baccepted_edge_id=edge-comparison-class-first-time-passers; accepted_dimension=comparison-class; from=lineage-ncbe-comparison-root,lineage-new-york-pass-rate-root; to=lineage-martinez-analysis-root; effects=Comparison populations are not interchangeable. | Administration mix can change percentile without new model evidence. | It parameterizes the comparison model rather than testing GPT-4. | The threshold defines a modeled comparator boundary.
edge-data-reported-score-reuselineage-model-performance-root → lineage-martinez-analysis-rootem:dossier-evidence-relation:sha256:691d48bc190063d50b6e811c7455ec325b1f29bcc7452301694470e83211188aaccepted_edge_id=edge-data-reported-score-reuse; accepted_dimension=data; from=lineage-model-performance-root; to=lineage-martinez-analysis-root; effects=The re-analysis is not a new model-performance experiment. | Score validation changes warrant, not the participant or model-performance root count.
edge-derivation-comparison-inputs--1lineage-ncbe-comparison-root → lineage-martinez-analysis-rootem:dossier-evidence-relation:sha256:8e1e29f851db71dfa10a5e236ce568d68a73d9c9dcc9e57be569669606ec66a7accepted_edge_id=edge-derivation-comparison-inputs; accepted_dimension=derivation; from=lineage-ncbe-comparison-root,lineage-new-york-pass-rate-root; to=lineage-martinez-analysis-root; effects=The value is a modeled input, not a GPT-4 observation. | Executable transformations do not create new source or performance roots.
edge-derivation-comparison-inputs--2lineage-new-york-pass-rate-root → lineage-martinez-analysis-rootem:dossier-evidence-relation:sha256:4cb023d071772f2c5de160eda970a23d215bd0e51de0ebef48513742b7b92ab4accepted_edge_id=edge-derivation-comparison-inputs; accepted_dimension=derivation; from=lineage-ncbe-comparison-root,lineage-new-york-pass-rate-root; to=lineage-martinez-analysis-root; effects=The value is a modeled input, not a GPT-4 observation. | Executable transformations do not create new source or performance roots.

Relation: relation-assertion-derive-martinez-first-time-essay em:dossier-evidence-relation:sha256:13748298644aa578769bbabfff2cef7491504ee79bf1d180ece2b059135ed552
Proposition: em:dossier-proposition:sha256:8b4297b5104a65c7f8bddf8b32ed22ff861e9c48bfd3f310cd2004f0422891a3
Assertion: em:dossier-assertion:sha256:cd35ba2db639224fb309527b6345a677d99b6477c0af28d3b1780b3b760db33e

{
  "derivation": {
    "comparison_population": "modeled first-time essay scores",
    "depends_on": [
      "derive-martinez-parameters"
    ],
    "derivation_id": "derive-martinez-first-time-essay",
    "method": "normal CDF at essay score 140 using assumed MBE distribution",
    "result_percentile": 41.520892709392534
  },
  "resolved_input_cells": []
}

span-calculation-derive-martinez-first-time-essayaccepted derivation and input cells: derive-martinez-first-time-essayem:dossier-span:sha256:7fe07e6a8c3bfc7145de80e008268fc0c0e4c52008cdcc2c56229e50818ae891 · sha256:a9c27ee78ab735acd0cc44a48abf93543d47b7e31106e8b749ad9acd73d94272em:dossier-edition:sha256:32d3146687af86ae7865d08fbc1636a0c0aa6195e52cb00350ce4b0d525ade2c · sha256:ab330f3d40dfdb5e467da1f66c8f2d318b6b5ed88b770555956d3c5c79151a4bem:dossier-source-work:sha256:bb86be7b07b85a8d09a4fab4b2526a49c82b959ee3fb31bcf33ff7a5e6e27831 · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions
span-martinez-mean-assumption§3.1, journal p. 588em:dossier-span:sha256:ecc711638dfc4b30115783c7b8d7f32b292b8c723a36f8dfbf421e41923b9461 · sha256:d292fbd6a0a2c16559d69c651b273374cef4c7383b7c2605b97f85381bf8c62dem:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667 · sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-july-mbe-distributionlines 55-58em:dossier-span:sha256:c93f04ab5c92ae7955b75278cff1f0392210fc5462823dbe30cba42b5ffbab38 · sha256:9912e2d4f1e369b4f9529d27b6d1fef5936814f0c6151a9fe9f675af8c375e0cem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-ube-sdlines 104-112em:dossier-span:sha256:7c297fee53ebcae4ddc2b4bd2301d9742b570914c364b3993890c92c3ab42d83 · sha256:29d389334aa2b12045b1d9febb630b66d80d832615d0764534e4e0a2ac49b55bem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-ncbe-ny-cutoff-2022Minimum Passing UBE Score by Jurisdiction in 2022, row 266em:dossier-span:sha256:160241d336f3cc06be5b4d52f6ac9229563916a9ca197dbbacb968b7c60981f8 · sha256:8e029f660745d0cb0de2acd77e8d6e8c8f9c544d1ba560c3250bc30861ad653bem:dossier-edition:sha256:2000492e8e5040cd8f5eaa216e7fc1d4d21d4f58afbfc6af5a218f4f53333081 · sha256:b55af410679b66b8a0057d5f8dd17baec7985675c94d7af44818f5c4c418cfb7em:dossier-source-work:sha256:324982bb2b90295b84f9e309ab84ca9a46cd0f1be9e9e3079a18bcc20bf58e63 · no open license confirmed · no open license confirmed
span-ny-first-timers-2022FIRST TIMERS, ALL Candidates, Combined 2022em:dossier-span:sha256:468763dc341321dd638ef9a08f6606ac5e02806a4086e4b0ba7bc05400c1d703 · sha256:e55680bc3d09d5c2968a8449c122db232a89c5f89b29dc9944b69c2b1eeeee11em:dossier-edition:sha256:10bbf9ff899ba3ad12de154e0ab9ea7cc3f34c0f2263e79951e26539911bec5f · sha256:bbdcb28fb38b0d7d03ddb4e4bcfbcca170d2443491a180fee0978c7510cdb74fem:dossier-source-work:sha256:51aa3262776f5da4a8ef12ab7f49366ef761951213e3a38f519a0a6b5d6534f1 · no open license confirmed · no open license confirmed
span-reshetar-first-time-meanparagraph beginning 'The numbers from 2019'em:dossier-span:sha256:6f0efeaf8fa67729f3f75793e1b64ce9961bb3463e5acc9d9c4a1d5aeb588520 · sha256:1452bea6f3158b2b58d90800d02a6cf2348935e6a194e07ab22efc88c5f26302em:dossier-edition:sha256:852a2ad23d9f6f760f0f21c4147eb6dd1b3c936ea9704e3dc3f743de193e4e49 · sha256:460f8d22d10526ee2a92e1fb4045e1cabacb0dc4a668e142a91a30592ccf0ed2em:dossier-source-work:sha256:802fad21dc2e69f5baff90d345e036b898bf958351af8eae39f346d23e005a12 · no open license confirmed · no open license confirmed

Calculation · normal CDF at MBE score 158: 78.82324690739395; equation=null; comparison_population="modeled first-time MBE takers"; depends_on=["derive-martinez-parameters"].

Relation: relation-assertion-derive-martinez-first-time-mbe em:dossier-evidence-relation:sha256:e7b1110a59e6ed3c7a511c4e1e77aa7664a34e076f572963e8d3790a981334bc
Proposition: em:dossier-proposition:sha256:added6c64552253aad1f336b1c4d88de6dd6172647d50fa5bddd3820feecf458
Assertion: em:dossier-assertion:sha256:7df08eee7d48ee98980dc48dcd4257c04f6a03ee5db70b69f16762e4122880ff

{
  "derivation": {
    "comparison_population": "modeled first-time MBE takers",
    "depends_on": [
      "derive-martinez-parameters"
    ],
    "derivation_id": "derive-martinez-first-time-mbe",
    "method": "normal CDF at MBE score 158",
    "result_percentile": 78.82324690739395
  },
  "resolved_input_cells": []
}

span-calculation-derive-martinez-first-time-mbeaccepted derivation and input cells: derive-martinez-first-time-mbeem:dossier-span:sha256:46a2e94c6b7fa7049cb7e26fb0def120ac9445a8bd95c1b21478c0db1e94424a · sha256:0222f9c217d52a28eaeb28caf33d560864fd14ba99c48d7ebd7fa0aa4fdba995em:dossier-edition:sha256:32d3146687af86ae7865d08fbc1636a0c0aa6195e52cb00350ce4b0d525ade2c · sha256:ab330f3d40dfdb5e467da1f66c8f2d318b6b5ed88b770555956d3c5c79151a4bem:dossier-source-work:sha256:bb86be7b07b85a8d09a4fab4b2526a49c82b959ee3fb31bcf33ff7a5e6e27831 · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions
span-martinez-mean-assumption§3.1, journal p. 588em:dossier-span:sha256:ecc711638dfc4b30115783c7b8d7f32b292b8c723a36f8dfbf421e41923b9461 · sha256:d292fbd6a0a2c16559d69c651b273374cef4c7383b7c2605b97f85381bf8c62dem:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667 · sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-july-mbe-distributionlines 55-58em:dossier-span:sha256:c93f04ab5c92ae7955b75278cff1f0392210fc5462823dbe30cba42b5ffbab38 · sha256:9912e2d4f1e369b4f9529d27b6d1fef5936814f0c6151a9fe9f675af8c375e0cem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-ube-sdlines 104-112em:dossier-span:sha256:7c297fee53ebcae4ddc2b4bd2301d9742b570914c364b3993890c92c3ab42d83 · sha256:29d389334aa2b12045b1d9febb630b66d80d832615d0764534e4e0a2ac49b55bem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-ncbe-ny-cutoff-2022Minimum Passing UBE Score by Jurisdiction in 2022, row 266em:dossier-span:sha256:160241d336f3cc06be5b4d52f6ac9229563916a9ca197dbbacb968b7c60981f8 · sha256:8e029f660745d0cb0de2acd77e8d6e8c8f9c544d1ba560c3250bc30861ad653bem:dossier-edition:sha256:2000492e8e5040cd8f5eaa216e7fc1d4d21d4f58afbfc6af5a218f4f53333081 · sha256:b55af410679b66b8a0057d5f8dd17baec7985675c94d7af44818f5c4c418cfb7em:dossier-source-work:sha256:324982bb2b90295b84f9e309ab84ca9a46cd0f1be9e9e3079a18bcc20bf58e63 · no open license confirmed · no open license confirmed
span-ny-first-timers-2022FIRST TIMERS, ALL Candidates, Combined 2022em:dossier-span:sha256:468763dc341321dd638ef9a08f6606ac5e02806a4086e4b0ba7bc05400c1d703 · sha256:e55680bc3d09d5c2968a8449c122db232a89c5f89b29dc9944b69c2b1eeeee11em:dossier-edition:sha256:10bbf9ff899ba3ad12de154e0ab9ea7cc3f34c0f2263e79951e26539911bec5f · sha256:bbdcb28fb38b0d7d03ddb4e4bcfbcca170d2443491a180fee0978c7510cdb74fem:dossier-source-work:sha256:51aa3262776f5da4a8ef12ab7f49366ef761951213e3a38f519a0a6b5d6534f1 · no open license confirmed · no open license confirmed
span-reshetar-first-time-meanparagraph beginning 'The numbers from 2019'em:dossier-span:sha256:6f0efeaf8fa67729f3f75793e1b64ce9961bb3463e5acc9d9c4a1d5aeb588520 · sha256:1452bea6f3158b2b58d90800d02a6cf2348935e6a194e07ab22efc88c5f26302em:dossier-edition:sha256:852a2ad23d9f6f760f0f21c4147eb6dd1b3c936ea9704e3dc3f743de193e4e49 · sha256:460f8d22d10526ee2a92e1fb4045e1cabacb0dc4a668e142a91a30592ccf0ed2em:dossier-source-work:sha256:802fad21dc2e69f5baff90d345e036b898bf958351af8eae39f346d23e005a12 · no open license confirmed · no open license confirmed

Relation: relation-assertion-derive-martinez-first-time-ube em:dossier-evidence-relation:sha256:a2c43a47e3f6b92d24687c9aa8fc109e6c1395031b0d99748789cacc006a0718
Proposition: em:dossier-proposition:sha256:e239947e9f182874d45854038dd31c8374fadebae4638ef0e25f91f286fe7202
Assertion: em:dossier-assertion:sha256:a5fe847c3d60f4185c2335a2160eb3e8223fb2e1a8e2174b611efd1cc9f442d0

{
  "derivation": {
    "comparison_population": "modeled first-time UBE takers",
    "depends_on": [
      "derive-martinez-parameters"
    ],
    "derivation_id": "derive-martinez-first-time-ube",
    "equation": "100 * Phi((298 - 287.6) / derived_ube_sd)",
    "method": "normal CDF at score 298",
    "result_percentile": 61.60252541656707
  },
  "resolved_input_cells": []
}

span-calculation-derive-martinez-first-time-ubeaccepted derivation and input cells: derive-martinez-first-time-ubeem:dossier-span:sha256:282eac343e251f80212871bfab6bbad216ef94e8989d5f56387cc19a79d72aca · sha256:95f5718ef7d21ae6fb0218ba919638f6bee7e0a849abfab1144857c83941d8c2em:dossier-edition:sha256:32d3146687af86ae7865d08fbc1636a0c0aa6195e52cb00350ce4b0d525ade2c · sha256:ab330f3d40dfdb5e467da1f66c8f2d318b6b5ed88b770555956d3c5c79151a4bem:dossier-source-work:sha256:bb86be7b07b85a8d09a4fab4b2526a49c82b959ee3fb31bcf33ff7a5e6e27831 · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions
span-martinez-mean-assumption§3.1, journal p. 588em:dossier-span:sha256:ecc711638dfc4b30115783c7b8d7f32b292b8c723a36f8dfbf421e41923b9461 · sha256:d292fbd6a0a2c16559d69c651b273374cef4c7383b7c2605b97f85381bf8c62dem:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667 · sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-july-mbe-distributionlines 55-58em:dossier-span:sha256:c93f04ab5c92ae7955b75278cff1f0392210fc5462823dbe30cba42b5ffbab38 · sha256:9912e2d4f1e369b4f9529d27b6d1fef5936814f0c6151a9fe9f675af8c375e0cem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-ube-sdlines 104-112em:dossier-span:sha256:7c297fee53ebcae4ddc2b4bd2301d9742b570914c364b3993890c92c3ab42d83 · sha256:29d389334aa2b12045b1d9febb630b66d80d832615d0764534e4e0a2ac49b55bem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-ncbe-ny-cutoff-2022Minimum Passing UBE Score by Jurisdiction in 2022, row 266em:dossier-span:sha256:160241d336f3cc06be5b4d52f6ac9229563916a9ca197dbbacb968b7c60981f8 · sha256:8e029f660745d0cb0de2acd77e8d6e8c8f9c544d1ba560c3250bc30861ad653bem:dossier-edition:sha256:2000492e8e5040cd8f5eaa216e7fc1d4d21d4f58afbfc6af5a218f4f53333081 · sha256:b55af410679b66b8a0057d5f8dd17baec7985675c94d7af44818f5c4c418cfb7em:dossier-source-work:sha256:324982bb2b90295b84f9e309ab84ca9a46cd0f1be9e9e3079a18bcc20bf58e63 · no open license confirmed · no open license confirmed
span-ny-first-timers-2022FIRST TIMERS, ALL Candidates, Combined 2022em:dossier-span:sha256:468763dc341321dd638ef9a08f6606ac5e02806a4086e4b0ba7bc05400c1d703 · sha256:e55680bc3d09d5c2968a8449c122db232a89c5f89b29dc9944b69c2b1eeeee11em:dossier-edition:sha256:10bbf9ff899ba3ad12de154e0ab9ea7cc3f34c0f2263e79951e26539911bec5f · sha256:bbdcb28fb38b0d7d03ddb4e4bcfbcca170d2443491a180fee0978c7510cdb74fem:dossier-source-work:sha256:51aa3262776f5da4a8ef12ab7f49366ef761951213e3a38f519a0a6b5d6534f1 · no open license confirmed · no open license confirmed
span-reshetar-first-time-meanparagraph beginning 'The numbers from 2019'em:dossier-span:sha256:6f0efeaf8fa67729f3f75793e1b64ce9961bb3463e5acc9d9c4a1d5aeb588520 · sha256:1452bea6f3158b2b58d90800d02a6cf2348935e6a194e07ab22efc88c5f26302em:dossier-edition:sha256:852a2ad23d9f6f760f0f21c4147eb6dd1b3c936ea9704e3dc3f743de193e4e49 · sha256:460f8d22d10526ee2a92e1fb4045e1cabacb0dc4a668e142a91a30592ccf0ed2em:dossier-source-work:sha256:802fad21dc2e69f5baff90d345e036b898bf958351af8eae39f346d23e005a12 · no open license confirmed · no open license confirmed

Relation: relation-assertion-derive-martinez-parameters em:dossier-evidence-relation:sha256:592ad3c88ad3d542736bb92ba9aeab1c3f375fd40d9688295eb6e8021a7d88e7
Proposition: em:dossier-proposition:sha256:8a265341b3cca87de499aa7fe9807fcb85405a680c9899fb2844a96717923ac6
Assertion: em:dossier-assertion:sha256:46b0fa65ec0a5d666de7e82ebfe7ea040c7bccf9d1862d67389fdc9ae6f6d6cb

{
  "derivation": {
    "derivation_id": "derive-martinez-parameters",
    "input_cell_ids": [
      "cell-martinez-july-mbe-85",
      "cell-martinez-july-mbe-90",
      "cell-martinez-july-mbe-95",
      "cell-martinez-july-mbe-100",
      "cell-martinez-july-mbe-105",
      "cell-martinez-july-mbe-110",
      "cell-martinez-july-mbe-115",
      "cell-martinez-july-mbe-120",
      "cell-martinez-july-mbe-125",
      "cell-martinez-july-mbe-130",
      "cell-martinez-july-mbe-135",
      "cell-martinez-july-mbe-140",
      "cell-martinez-july-mbe-145",
      "cell-martinez-july-mbe-150",
      "cell-martinez-july-mbe-155",
      "cell-martinez-july-mbe-160",
      "cell-martinez-july-mbe-165",
      "cell-martinez-july-mbe-170",
      "cell-martinez-july-mbe-175",
      "cell-martinez-july-mbe-180",
      "cell-martinez-july-mbe-185",
      "cell-ncbe-ube-score-266",
      "cell-ny-first-timers-rate"
    ],
    "input_span_ids": [
      "span-reshetar-first-time-mean",
      "span-martinez-mean-assumption",
      "span-martinez-script-july-mbe-distribution",
      "span-martinez-script-ube-sd",
      "span-ncbe-ny-cutoff-2022",
      "span-ny-first-timers-2022"
    ],
    "inputs": {
      "assumed_first_time_essay_mean": 143.8,
      "assumed_first_time_ube_mean": 287.6,
      "first_time_mbe_mean": 143.8,
      "july_mbe_binned_observations": 1002,
      "new_york_cutoff": 266.0,
      "new_york_nonpass_proportion": 0.27
    },
    "method": "analytic reproduction of the executable OSF inputs",
    "results": {
      "derived_ube_sd": 35.24729455256271,
      "sample_mbe_sd": 17.74327194332649,
      "z_at_0_27": -0.6128129910166272
    },
    "uncertainty": "The UBE distribution is inferred from aggregate inputs and normality; the essay mean/SD are assumed rather than observed."
  },
  "resolved_input_cells": [
    {
      "cell_id": "cell-martinez-july-mbe-85",
      "count": 2,
      "score": 85
    },
    {
      "cell_id": "cell-martinez-july-mbe-90",
      "count": 2,
      "score": 90
    },
    {
      "cell_id": "cell-martinez-july-mbe-95",
      "count": 5,
      "score": 95
    },
    {
      "cell_id": "cell-martinez-july-mbe-100",
      "count": 6,
      "score": 100
    },
    {
      "cell_id": "cell-martinez-july-mbe-105",
      "count": 13,
      "score": 105
    },
    {
      "cell_id": "cell-martinez-july-mbe-110",
      "count": 22,
      "score": 110
    },
    {
      "cell_id": "cell-martinez-july-mbe-115",
      "count": 33,
      "score": 115
    },
    {
      "cell_id": "cell-martinez-july-mbe-120",
      "count": 56,
      "score": 120
    },
    {
      "cell_id": "cell-martinez-july-mbe-125",
      "count": 73,
      "score": 125
    },
    {
      "cell_id": "cell-martinez-july-mbe-130",
      "count": 78,
      "score": 130
    },
    {
      "cell_id": "cell-martinez-july-mbe-135",
      "count": 104,
      "score": 135
    },
    {
      "cell_id": "cell-martinez-july-mbe-140",
      "count": 96,
      "score": 140
    },
    {
      "cell_id": "cell-martinez-july-mbe-145",
      "count": 101,
      "score": 145
    },
    {
      "cell_id": "cell-martinez-july-mbe-150",
      "count": 99,
      "score": 150
    },
    {
      "cell_id": "cell-martinez-july-mbe-155",
      "count": 99,
      "score": 155
    },
    {
      "cell_id": "cell-martinez-july-mbe-160",
      "count": 79,
      "score": 160
    },
    {
      "cell_id": "cell-martinez-july-mbe-165",
      "count": 64,
      "score": 165
    },
    {
      "cell_id": "cell-martinez-july-mbe-170",
      "count": 38,
      "score": 170
    },
    {
      "cell_id": "cell-martinez-july-mbe-175",
      "count": 22,
      "score": 175
    },
    {
      "cell_id": "cell-martinez-july-mbe-180",
      "count": 8,
      "score": 180
    },
    {
      "cell_id": "cell-martinez-july-mbe-185",
      "count": 2,
      "score": 185
    },
    {
      "cell_id": "cell-ncbe-ube-score-266",
      "column": "Score",
      "row": "266",
      "text": "266"
    },
    {
      "cell_id": "cell-ny-first-timers-rate",
      "column": "Rate",
      "row": "ALL Candidates",
      "text": "73%"
    }
  ]
}

span-calculation-derive-martinez-parametersaccepted derivation and input cells: derive-martinez-parametersem:dossier-span:sha256:b36533d6f13983fa968a987d5263ea1ab6220a87e2b416ab59a0c3d40ea03d09 · sha256:b92c0e5f9e2866c3806a45f478310c538513bb17644f78c91900edf5fc547d2eem:dossier-edition:sha256:32d3146687af86ae7865d08fbc1636a0c0aa6195e52cb00350ce4b0d525ade2c · sha256:ab330f3d40dfdb5e467da1f66c8f2d318b6b5ed88b770555956d3c5c79151a4bem:dossier-source-work:sha256:bb86be7b07b85a8d09a4fab4b2526a49c82b959ee3fb31bcf33ff7a5e6e27831 · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions
span-martinez-mean-assumption§3.1, journal p. 588em:dossier-span:sha256:ecc711638dfc4b30115783c7b8d7f32b292b8c723a36f8dfbf421e41923b9461 · sha256:d292fbd6a0a2c16559d69c651b273374cef4c7383b7c2605b97f85381bf8c62dem:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667 · sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-july-mbe-distributionlines 55-58em:dossier-span:sha256:c93f04ab5c92ae7955b75278cff1f0392210fc5462823dbe30cba42b5ffbab38 · sha256:9912e2d4f1e369b4f9529d27b6d1fef5936814f0c6151a9fe9f675af8c375e0cem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-ube-sdlines 104-112em:dossier-span:sha256:7c297fee53ebcae4ddc2b4bd2301d9742b570914c364b3993890c92c3ab42d83 · sha256:29d389334aa2b12045b1d9febb630b66d80d832615d0764534e4e0a2ac49b55bem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-ncbe-ny-cutoff-2022Minimum Passing UBE Score by Jurisdiction in 2022, row 266em:dossier-span:sha256:160241d336f3cc06be5b4d52f6ac9229563916a9ca197dbbacb968b7c60981f8 · sha256:8e029f660745d0cb0de2acd77e8d6e8c8f9c544d1ba560c3250bc30861ad653bem:dossier-edition:sha256:2000492e8e5040cd8f5eaa216e7fc1d4d21d4f58afbfc6af5a218f4f53333081 · sha256:b55af410679b66b8a0057d5f8dd17baec7985675c94d7af44818f5c4c418cfb7em:dossier-source-work:sha256:324982bb2b90295b84f9e309ab84ca9a46cd0f1be9e9e3079a18bcc20bf58e63 · no open license confirmed · no open license confirmed
span-ny-first-timers-2022FIRST TIMERS, ALL Candidates, Combined 2022em:dossier-span:sha256:468763dc341321dd638ef9a08f6606ac5e02806a4086e4b0ba7bc05400c1d703 · sha256:e55680bc3d09d5c2968a8449c122db232a89c5f89b29dc9944b69c2b1eeeee11em:dossier-edition:sha256:10bbf9ff899ba3ad12de154e0ab9ea7cc3f34c0f2263e79951e26539911bec5f · sha256:bbdcb28fb38b0d7d03ddb4e4bcfbcca170d2443491a180fee0978c7510cdb74fem:dossier-source-work:sha256:51aa3262776f5da4a8ef12ab7f49366ef761951213e3a38f519a0a6b5d6534f1 · no open license confirmed · no open license confirmed
span-reshetar-first-time-meanparagraph beginning 'The numbers from 2019'em:dossier-span:sha256:6f0efeaf8fa67729f3f75793e1b64ce9961bb3463e5acc9d9c4a1d5aeb588520 · sha256:1452bea6f3158b2b58d90800d02a6cf2348935e6a194e07ab22efc88c5f26302em:dossier-edition:sha256:852a2ad23d9f6f760f0f21c4147eb6dd1b3c936ea9704e3dc3f743de193e4e49 · sha256:460f8d22d10526ee2a92e1fb4045e1cabacb0dc4a668e142a91a30592ccf0ed2em:dossier-source-work:sha256:802fad21dc2e69f5baff90d345e036b898bf958351af8eae39f346d23e005a12 · no open license confirmed · no open license confirmed

Relation: relation-assertion-derive-martinez-passers-essay em:dossier-evidence-relation:sha256:ab1d937208a7dd4488c51004af267a03414306012ee327ecf10c05d0645a32b7
Proposition: em:dossier-proposition:sha256:6c97aaa7a9bc70f03c07f91df685245b1a1278b97bb4e9bcf56f7e85861dda76
Assertion: em:dossier-assertion:sha256:bfe2fd3438fb2f351efa67ef5066876074b428fd1fcd8b6c379070873fda676b

{
  "derivation": {
    "comparison_population": "modeled essay scores at or above 135",
    "depends_on": [
      "derive-martinez-parameters"
    ],
    "derivation_id": "derive-martinez-passers-essay",
    "method": "normal CDF conditional on modeled essay score >= 135",
    "result_percentile": 15.252536216882072
  },
  "resolved_input_cells": []
}

span-calculation-derive-martinez-passers-essayaccepted derivation and input cells: derive-martinez-passers-essayem:dossier-span:sha256:75c94b1ed032d669e53b051fa98d8234be91d8062a23ee43147ac17553d9e5db · sha256:e28cf7118b36578f51acca1366d26808d9189b9b699ff2f8ce7c4b77bde9b468em:dossier-edition:sha256:32d3146687af86ae7865d08fbc1636a0c0aa6195e52cb00350ce4b0d525ade2c · sha256:ab330f3d40dfdb5e467da1f66c8f2d318b6b5ed88b770555956d3c5c79151a4bem:dossier-source-work:sha256:bb86be7b07b85a8d09a4fab4b2526a49c82b959ee3fb31bcf33ff7a5e6e27831 · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions
span-martinez-mean-assumption§3.1, journal p. 588em:dossier-span:sha256:ecc711638dfc4b30115783c7b8d7f32b292b8c723a36f8dfbf421e41923b9461 · sha256:d292fbd6a0a2c16559d69c651b273374cef4c7383b7c2605b97f85381bf8c62dem:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667 · sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-july-mbe-distributionlines 55-58em:dossier-span:sha256:c93f04ab5c92ae7955b75278cff1f0392210fc5462823dbe30cba42b5ffbab38 · sha256:9912e2d4f1e369b4f9529d27b6d1fef5936814f0c6151a9fe9f675af8c375e0cem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-ube-sdlines 104-112em:dossier-span:sha256:7c297fee53ebcae4ddc2b4bd2301d9742b570914c364b3993890c92c3ab42d83 · sha256:29d389334aa2b12045b1d9febb630b66d80d832615d0764534e4e0a2ac49b55bem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-ncbe-ny-cutoff-2022Minimum Passing UBE Score by Jurisdiction in 2022, row 266em:dossier-span:sha256:160241d336f3cc06be5b4d52f6ac9229563916a9ca197dbbacb968b7c60981f8 · sha256:8e029f660745d0cb0de2acd77e8d6e8c8f9c544d1ba560c3250bc30861ad653bem:dossier-edition:sha256:2000492e8e5040cd8f5eaa216e7fc1d4d21d4f58afbfc6af5a218f4f53333081 · sha256:b55af410679b66b8a0057d5f8dd17baec7985675c94d7af44818f5c4c418cfb7em:dossier-source-work:sha256:324982bb2b90295b84f9e309ab84ca9a46cd0f1be9e9e3079a18bcc20bf58e63 · no open license confirmed · no open license confirmed
span-ny-first-timers-2022FIRST TIMERS, ALL Candidates, Combined 2022em:dossier-span:sha256:468763dc341321dd638ef9a08f6606ac5e02806a4086e4b0ba7bc05400c1d703 · sha256:e55680bc3d09d5c2968a8449c122db232a89c5f89b29dc9944b69c2b1eeeee11em:dossier-edition:sha256:10bbf9ff899ba3ad12de154e0ab9ea7cc3f34c0f2263e79951e26539911bec5f · sha256:bbdcb28fb38b0d7d03ddb4e4bcfbcca170d2443491a180fee0978c7510cdb74fem:dossier-source-work:sha256:51aa3262776f5da4a8ef12ab7f49366ef761951213e3a38f519a0a6b5d6534f1 · no open license confirmed · no open license confirmed
span-reshetar-first-time-meanparagraph beginning 'The numbers from 2019'em:dossier-span:sha256:6f0efeaf8fa67729f3f75793e1b64ce9961bb3463e5acc9d9c4a1d5aeb588520 · sha256:1452bea6f3158b2b58d90800d02a6cf2348935e6a194e07ab22efc88c5f26302em:dossier-edition:sha256:852a2ad23d9f6f760f0f21c4147eb6dd1b3c936ea9704e3dc3f743de193e4e49 · sha256:460f8d22d10526ee2a92e1fb4045e1cabacb0dc4a668e142a91a30592ccf0ed2em:dossier-source-work:sha256:802fad21dc2e69f5baff90d345e036b898bf958351af8eae39f346d23e005a12 · no open license confirmed · no open license confirmed

Relation: relation-assertion-derive-martinez-passers-mbe em:dossier-evidence-relation:sha256:1dd0e17278567855ddd02c50c7a61cc75f28fc83737928dcc68e610cce4ba6b2
Proposition: em:dossier-proposition:sha256:c564ce54eb3b2d6f72781fee0e99f5162aec79993a6aa87f999d7b8d2775420e
Assertion: em:dossier-assertion:sha256:837a7e443196825e489e5e7f7a09e74cdcd36ce65a43cf63c504a479c59da0a5

{
  "derivation": {
    "comparison_population": "modeled MBE scores at or above 135",
    "depends_on": [
      "derive-martinez-parameters"
    ],
    "derivation_id": "derive-martinez-passers-mbe",
    "method": "normal CDF conditional on modeled MBE score >= 135",
    "result_percentile": 69.31081545343784
  },
  "resolved_input_cells": []
}

span-calculation-derive-martinez-passers-mbeaccepted derivation and input cells: derive-martinez-passers-mbeem:dossier-span:sha256:25dbcaa16173c2875dcab51c6fbd9210df50cf2aa03cb3e2514724edcc667200 · sha256:bc34a7eda1879f438d7e7945440d3db419371f000787488a1830b7c283f81845em:dossier-edition:sha256:32d3146687af86ae7865d08fbc1636a0c0aa6195e52cb00350ce4b0d525ade2c · sha256:ab330f3d40dfdb5e467da1f66c8f2d318b6b5ed88b770555956d3c5c79151a4bem:dossier-source-work:sha256:bb86be7b07b85a8d09a4fab4b2526a49c82b959ee3fb31bcf33ff7a5e6e27831 · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions
span-martinez-mean-assumption§3.1, journal p. 588em:dossier-span:sha256:ecc711638dfc4b30115783c7b8d7f32b292b8c723a36f8dfbf421e41923b9461 · sha256:d292fbd6a0a2c16559d69c651b273374cef4c7383b7c2605b97f85381bf8c62dem:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667 · sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-july-mbe-distributionlines 55-58em:dossier-span:sha256:c93f04ab5c92ae7955b75278cff1f0392210fc5462823dbe30cba42b5ffbab38 · sha256:9912e2d4f1e369b4f9529d27b6d1fef5936814f0c6151a9fe9f675af8c375e0cem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-ube-sdlines 104-112em:dossier-span:sha256:7c297fee53ebcae4ddc2b4bd2301d9742b570914c364b3993890c92c3ab42d83 · sha256:29d389334aa2b12045b1d9febb630b66d80d832615d0764534e4e0a2ac49b55bem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-ncbe-ny-cutoff-2022Minimum Passing UBE Score by Jurisdiction in 2022, row 266em:dossier-span:sha256:160241d336f3cc06be5b4d52f6ac9229563916a9ca197dbbacb968b7c60981f8 · sha256:8e029f660745d0cb0de2acd77e8d6e8c8f9c544d1ba560c3250bc30861ad653bem:dossier-edition:sha256:2000492e8e5040cd8f5eaa216e7fc1d4d21d4f58afbfc6af5a218f4f53333081 · sha256:b55af410679b66b8a0057d5f8dd17baec7985675c94d7af44818f5c4c418cfb7em:dossier-source-work:sha256:324982bb2b90295b84f9e309ab84ca9a46cd0f1be9e9e3079a18bcc20bf58e63 · no open license confirmed · no open license confirmed
span-ny-first-timers-2022FIRST TIMERS, ALL Candidates, Combined 2022em:dossier-span:sha256:468763dc341321dd638ef9a08f6606ac5e02806a4086e4b0ba7bc05400c1d703 · sha256:e55680bc3d09d5c2968a8449c122db232a89c5f89b29dc9944b69c2b1eeeee11em:dossier-edition:sha256:10bbf9ff899ba3ad12de154e0ab9ea7cc3f34c0f2263e79951e26539911bec5f · sha256:bbdcb28fb38b0d7d03ddb4e4bcfbcca170d2443491a180fee0978c7510cdb74fem:dossier-source-work:sha256:51aa3262776f5da4a8ef12ab7f49366ef761951213e3a38f519a0a6b5d6534f1 · no open license confirmed · no open license confirmed
span-reshetar-first-time-meanparagraph beginning 'The numbers from 2019'em:dossier-span:sha256:6f0efeaf8fa67729f3f75793e1b64ce9961bb3463e5acc9d9c4a1d5aeb588520 · sha256:1452bea6f3158b2b58d90800d02a6cf2348935e6a194e07ab22efc88c5f26302em:dossier-edition:sha256:852a2ad23d9f6f760f0f21c4147eb6dd1b3c936ea9704e3dc3f743de193e4e49 · sha256:460f8d22d10526ee2a92e1fb4045e1cabacb0dc4a668e142a91a30592ccf0ed2em:dossier-source-work:sha256:802fad21dc2e69f5baff90d345e036b898bf958351af8eae39f346d23e005a12 · no open license confirmed · no open license confirmed

Relation: relation-assertion-derive-martinez-passers-ube em:dossier-evidence-relation:sha256:20f088f009065cfc3380add119f1dd245d0cc60c26ce7304d2603d810dcceac5
Proposition: em:dossier-proposition:sha256:9119f9ff62ea1e7760aabb53df19e6824e081aaab2d28ddac873b49ba1829c69
Assertion: em:dossier-assertion:sha256:0e84c467e4cdeee807106e9fa021760a1a2e63bcd7fb41c4cfccea488558b23f

{
  "derivation": {
    "comparison_population": "modeled first-time scores at or above 270",
    "depends_on": [
      "derive-martinez-parameters"
    ],
    "derivation_id": "derive-martinez-passers-ube",
    "equation": "100 * (F(298) - F(270)) / (1 - F(270))",
    "method": "normal CDF conditional on modeled UBE score >= 270",
    "result_percentile": 44.45020553281165,
    "uncertainty": "The script uses 270 for this filter after using New York's 266 cutoff to infer UBE SD."
  },
  "resolved_input_cells": []
}

span-calculation-derive-martinez-passers-ubeaccepted derivation and input cells: derive-martinez-passers-ubeem:dossier-span:sha256:3738518581952127b4be8dbb224e03a6eb3126a5ec5c685a837cba9e0d7bfcde · sha256:5cd325ee19d0d60ca21ed01dcf1cc3c6cfb1edfb2f9fb702d680f03a2ec1a8f3em:dossier-edition:sha256:32d3146687af86ae7865d08fbc1636a0c0aa6195e52cb00350ce4b0d525ade2c · sha256:ab330f3d40dfdb5e467da1f66c8f2d318b6b5ed88b770555956d3c5c79151a4bem:dossier-source-work:sha256:bb86be7b07b85a8d09a4fab4b2526a49c82b959ee3fb31bcf33ff7a5e6e27831 · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions · Repository instrumentation under Apache-2.0; accepted source licenses remain attached to their source editions
span-martinez-mean-assumption§3.1, journal p. 588em:dossier-span:sha256:ecc711638dfc4b30115783c7b8d7f32b292b8c723a36f8dfbf421e41923b9461 · sha256:d292fbd6a0a2c16559d69c651b273374cef4c7383b7c2605b97f85381bf8c62dem:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667 · sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-july-mbe-distributionlines 55-58em:dossier-span:sha256:c93f04ab5c92ae7955b75278cff1f0392210fc5462823dbe30cba42b5ffbab38 · sha256:9912e2d4f1e369b4f9529d27b6d1fef5936814f0c6151a9fe9f675af8c375e0cem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-martinez-script-ube-sdlines 104-112em:dossier-span:sha256:7c297fee53ebcae4ddc2b4bd2301d9742b570914c364b3993890c92c3ab42d83 · sha256:29d389334aa2b12045b1d9febb630b66d80d832615d0764534e4e0a2ac49b55bem:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745 · sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed
span-ncbe-ny-cutoff-2022Minimum Passing UBE Score by Jurisdiction in 2022, row 266em:dossier-span:sha256:160241d336f3cc06be5b4d52f6ac9229563916a9ca197dbbacb968b7c60981f8 · sha256:8e029f660745d0cb0de2acd77e8d6e8c8f9c544d1ba560c3250bc30861ad653bem:dossier-edition:sha256:2000492e8e5040cd8f5eaa216e7fc1d4d21d4f58afbfc6af5a218f4f53333081 · sha256:b55af410679b66b8a0057d5f8dd17baec7985675c94d7af44818f5c4c418cfb7em:dossier-source-work:sha256:324982bb2b90295b84f9e309ab84ca9a46cd0f1be9e9e3079a18bcc20bf58e63 · no open license confirmed · no open license confirmed
span-ny-first-timers-2022FIRST TIMERS, ALL Candidates, Combined 2022em:dossier-span:sha256:468763dc341321dd638ef9a08f6606ac5e02806a4086e4b0ba7bc05400c1d703 · sha256:e55680bc3d09d5c2968a8449c122db232a89c5f89b29dc9944b69c2b1eeeee11em:dossier-edition:sha256:10bbf9ff899ba3ad12de154e0ab9ea7cc3f34c0f2263e79951e26539911bec5f · sha256:bbdcb28fb38b0d7d03ddb4e4bcfbcca170d2443491a180fee0978c7510cdb74fem:dossier-source-work:sha256:51aa3262776f5da4a8ef12ab7f49366ef761951213e3a38f519a0a6b5d6534f1 · no open license confirmed · no open license confirmed
span-reshetar-first-time-meanparagraph beginning 'The numbers from 2019'em:dossier-span:sha256:6f0efeaf8fa67729f3f75793e1b64ce9961bb3463e5acc9d9c4a1d5aeb588520 · sha256:1452bea6f3158b2b58d90800d02a6cf2348935e6a194e07ab22efc88c5f26302em:dossier-edition:sha256:852a2ad23d9f6f760f0f21c4147eb6dd1b3c936ea9704e3dc3f743de193e4e49 · sha256:460f8d22d10526ee2a92e1fb4045e1cabacb0dc4a668e142a91a30592ccf0ed2em:dossier-source-work:sha256:802fad21dc2e69f5baff90d345e036b898bf958351af8eae39f346d23e005a12 · no open license confirmed · no open license confirmed

percentile_analysis_new.Rmd

discussion, journal p. 598

when examining only those who passed the exam, GPT-4’s performance is estimated to drop to ∼48th percentile overall, and ∼15th percentile on essays.

Work: em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b
Edition: em:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667
Edition digest: sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5
Span: em:dossier-span:sha256:0b68dabbe1d021424ed271a53217094910a82be36032ddf2fb7d527e519ac280
Span digest: sha256:d999f5238c2ce339081283354841c9949e5a6a71a5370ea28eb635484832a707
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed

percentile_analysis_new.Rmd

§3.2.2, journal p. 591

With regard to the aggregate UBE score, GPT-4 scored in the ∼45th percentile.

Work: em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b
Edition: em:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667
Edition digest: sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5
Span: em:dossier-span:sha256:e4d9982c357259d2e11c9d6e647569ca0338134240924a766792fbba54de7502
Span digest: sha256:94dbb34cb0b30aefe6ad46a54298be28df4bf1014c3a0d389f63e4615b3dc9bf
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed

percentile_analysis_new.Rmd

lines 166-168

{ "format": "code-segment-transcription", "locator": "lines 166-168", "normalization": "exact-code-text", "segments": [ { "segment_id": "segment-martinez-code-filter-ube", "text": "filtered_data_ube <- data_ube[data_ube >= 270]" }, { "segment_id": "segment-martinez-code-filter-mbe", "text": "filtered_data_mbe <- data_mbe[data_mbe >= 135]" }, { "segment_id": "segment-martinez-code-filter-essay", "text": "filtered_data_essay <- data_essay[data_essay >= 135]" } ], "span_id": "span-martinez-script-thresholds", "supports": "The passers analysis filters UBE at 270 even though the SD input uses New York's 266 cutoff." }

Work: em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b
Edition: em:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745
Edition digest: sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666
Span: em:dossier-span:sha256:7da66b96a0ba3914388da9f878232040b78dc3be4727b77afe738797c9e53c8b
Span digest: sha256:e4f31613c4c910062cc0c84109333b81d866660c9565c354ac91f5fc1f76716f
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed

percentile_analysis_new.Rmd

Table 3, journal p. 591

{ "cells": [ { "cell_id": "cell-martinez-july-ube", "column": "UBE", "row": "July test-takers", "text": "1st\u201368th" }, { "cell_id": "cell-martinez-first-timers-ube", "column": "UBE", "row": "All first-timers", "text": "2nd\u201362rd" }, { "cell_id": "cell-martinez-qualified-attorneys-ube", "column": "UBE", "row": "Qualified attorneys", "text": "0th\u201345th" } ], "format": "table-cell-transcription", "locator": "Table 3, journal p. 591", "normalization": "exact-cell-text", "span_id": "span-martinez-table-45", "supports": "The table reports 45th, not 48th, for the passers/qualified-attorneys comparison." }

Work: em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b
Edition: em:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667
Edition digest: sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5
Span: em:dossier-span:sha256:474fb98347f7e6e5fa04ed79707987e9ee63b657d6f1c5e2435843ec6a1cddbe
Span digest: sha256:200178847adc47281f699728e39c7ce7bb3f84704ed70de68b2e0c012f955be5
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed

Illinois February 2018 Bar Examination Percentile Equivalents

total-scale table, rows 300 and 290

{ "cells": [ { "cell_id": "cell-illinois-feb-2018-300", "percentile": 90, "score": 300 }, { "cell_id": "cell-illinois-feb-2018-290", "percentile": 85, "score": 290 } ], "format": "table-cell-transcription", "locator": "total-scale table, rows 300 and 290", "normalization": "exact-cell-text", "span_id": "span-illinois-feb-2018-anchors", "supports": "The chart brackets 298 but contains no printed 298 row or interpolation rule." }

Work: em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd
Edition: em:dossier-edition:sha256:9be0e724fc35f74c96e635c6263f8f44ef804c3406c34b268bb710b24228a368
Edition digest: sha256:44de7bd89cd4aeacee7b38d7077ff659e3baf552cb9b295401ec9b7229c04964
Span: em:dossier-span:sha256:56a011a09e4e7dd05afbc0309c94d4dcfa1ecf1d6fd3df6a6a6f65204c7ccf03
Span digest: sha256:b3b8f789db7571a4fd8ed73e5380f2847aae920172722b9f6f51eec96c62e9ce
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

Illinois February 2018 Bar Examination Percentile Equivalents

total-scale table, rows 300 and 290

{ "cells": [ { "cell_id": "cell-illinois-feb-2019-300", "percentile": 90, "score": 300 }, { "cell_id": "cell-illinois-feb-2019-290", "percentile": 83, "score": 290 } ], "format": "table-cell-transcription", "locator": "total-scale table, rows 300 and 290", "normalization": "exact-cell-text", "span_id": "span-illinois-feb-2019-anchors", "supports": "This later February chart also brackets 298 but supplies no interpolation rule." }

Work: em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd
Edition: em:dossier-edition:sha256:2c2badc4c75c58ca49007bda1f0c1fcad4b301ecd060bc905bd8a741acd80925
Edition digest: sha256:91185bd6263d73e5976b5b00f28fb9b62301be0bb0347b1d0a308c295f6a6151
Span: em:dossier-span:sha256:1e2736835309f364d7f8553b6e26a7c9802dce86c720c84fa2097f7e3a755ca3
Span digest: sha256:29f7ab38f7ff54e4576aa5b3c6fd310debd0520513f68fc3e21744af073923f8
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

Illinois February 2018 Bar Examination Percentile Equivalents

total-scale table, rows 300 and 290

{ "cells": [ { "cell_id": "cell-illinois-jul-2018-300", "percentile": 70, "score": 300 }, { "cell_id": "cell-illinois-jul-2018-290", "percentile": 59, "score": 290 } ], "format": "table-cell-transcription", "locator": "total-scale table, rows 300 and 290", "normalization": "exact-cell-text", "span_id": "span-illinois-jul-2018-anchors", "supports": "The July chart places the same score region far below the February chart." }

Work: em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd
Edition: em:dossier-edition:sha256:2b720aa3b7368249b079f4bc7d3de383efc590510bc33e1ed88db133e9293237
Edition digest: sha256:a9aa31983924ffec8d0e713ad29fc16a0e466557a57eba5061577afc4698bae6
Span: em:dossier-span:sha256:3ec4b97842a7b34ea0a2857a50da7f6a02eb1556351c7ec727b2cccd2743637c
Span digest: sha256:fe399dcc350821ac8ab546593931f1e34b24d811cad9062126e487a53783a783
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

Appendix for GPT-4 passes the Bar Exam

footnote 5

While we are not fully convinced of the methodological approach taken in some subsequent analysis [78], we do agree that it would be better to consider the raw 297 UBE as falling within a range between 68th and 90th percentile (depending on the precise state and timing of the exam administration).

Work: em:dossier-source-work:sha256:880443d56eafdb224149d953f7de48e7550d53f814008551895b180ecb4b203b
Edition: em:dossier-edition:sha256:21621ede6a23bb3f98867c0918e5d9808d6dced5ca3f271a3a52e561810f2f7b
Edition digest: sha256:9d10cfedd531e62d96ea27bb78b00a9148e6b5d8135807836c5982e65a42416e
Span: em:dossier-span:sha256:477c595f89b6efc4f4fac04d627e65d95daa7929d10f76d3486a96c6c8bf8b06
Span digest: sha256:7e376175b36f8e2809bc74e378c7354858289a689370b70f82df2965d32b9026
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree · CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree

First-Time Exam Takers and Repeaters in 2022

table note

NOTE: First-time exam and repeat test taker data supplied by the jurisdictions in this chart are based on those examinees’ testing experience in the reporting jurisdiction only and do not account for possible previous attempts at the bar examination in other jurisdictions.

Work: em:dossier-source-work:sha256:324982bb2b90295b84f9e309ab84ca9a46cd0f1be9e9e3079a18bcc20bf58e63
Edition: em:dossier-edition:sha256:b3dddcd76a7bbe208e296d4b9a13fc53e91cd14df2ae42ee1c91803569240eca
Edition digest: sha256:f42357f4f20e9db92a1a48755cb2f269123cb4ee93cecc042bff83d990989794
Span: em:dossier-span:sha256:f8fe10f696eb9dc37fa067a0b10401f5bf1e29589d6c6071cfaa44c15e59b183
Span digest: sha256:1feddb6e088dae626d7d1cc8f6a0207c19ac7e9f736830bf3a3ae4085e374a42
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

First-Time Exam Takers and Repeaters in 2022

Minimum Passing UBE Score by Jurisdiction in 2022, row 266

{ "cells": [ { "cell_id": "cell-ncbe-ube-score-266", "column": "Score", "row": "266", "text": "266" }, { "cell_id": "cell-ncbe-ube-jurisdictions-266", "column": "Jurisdictions", "row": "266", "text": "Connecticut; District of Columbia; Illinois; Iowa; Kansas; Kentucky; Maryland; Montana; New Jersey; New York; South Carolina; Virgin Islands" } ], "format": "table-cell-transcription", "locator": "Minimum Passing UBE Score by Jurisdiction in 2022, row 266", "normalization": "exact-cell-text", "span_id": "span-ncbe-ny-cutoff-2022", "supports": "New York's 2022 minimum passing UBE score was 266." }

Work: em:dossier-source-work:sha256:324982bb2b90295b84f9e309ab84ca9a46cd0f1be9e9e3079a18bcc20bf58e63
Edition: em:dossier-edition:sha256:2000492e8e5040cd8f5eaa216e7fc1d4d21d4f58afbfc6af5a218f4f53333081
Edition digest: sha256:b55af410679b66b8a0057d5f8dd17baec7985675c94d7af44818f5c4c418cfb7
Span: em:dossier-span:sha256:160241d336f3cc06be5b4d52f6ac9229563916a9ca197dbbacb968b7c60981f8
Span digest: sha256:8e029f660745d0cb0de2acd77e8d6e8c8f9c544d1ba560c3250bc30861ad653b
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

First-Time Exam Takers and Repeaters in 2022

NCBE MBE-based data, February and July 2022 totals

{ "format": "exact-segments", "locator": "NCBE MBE-based data, February and July 2022 totals", "normalization": "collapse-whitespace", "segments": [ { "segment_id": "segment-ncbe-february-repeaters-count", "text": "February likely repeaters taking: 11,289" }, { "segment_id": "segment-ncbe-february-repeaters-percent", "text": "68% of February 2022 examinees were likely repeaters" }, { "segment_id": "segment-ncbe-february-first-timers-count", "text": "February likely first-timers taking: 5,215" }, { "segment_id": "segment-ncbe-february-first-timers-percent", "text": "32% of February 2022 examinees were likely first-time takers" }, { "segment_id": "segment-ncbe-july-repeaters-count", "text": "July likely repeaters taking: 10,200" }, { "segment_id": "segment-ncbe-july-repeaters-percent", "text": "23% of July 2022 examinees were likely repeaters" }, { "segment_id": "segment-ncbe-july-first-timers-count", "text": "July likely first-timers taking: 34,505" }, { "segment_id": "segment-ncbe-july-first-timers-percent", "text": "77% of July 2021 examinees were likely first-time takers" } ], "span_id": "span-ncbe-snapshot-composition", "supports": "February and July have materially different inferred first-time/repeater composition." }

Work: em:dossier-source-work:sha256:324982bb2b90295b84f9e309ab84ca9a46cd0f1be9e9e3079a18bcc20bf58e63
Edition: em:dossier-edition:sha256:5e5c17ba424f84711ba55a14e9bf2b2846c2c05389280d7e21e1f1b2ad37810d
Edition digest: sha256:d2ce119a524286fa88bbaf4438c9052d77a5b728990db41cbc8cad70dc6740c4
Span: em:dossier-span:sha256:352571f2e61d157a8943ca6216ef23f38b6ae5cde1b55838d1a424bcc4ee9f09
Span digest: sha256:c389829939a67dd8c1d353a75cb1bdbc6823160e0582ba17bf83ba6d1e939dbc
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

New York Bar Exam 2022 Statistics

FIRST TIMERS, ALL Candidates, Combined 2022

{ "cells": [ { "cell_id": "cell-ny-first-timers-took", "column": "Took", "row": "ALL Candidates", "text": "9,457" }, { "cell_id": "cell-ny-first-timers-passed", "column": "Passed", "row": "ALL Candidates", "text": "6,867" }, { "cell_id": "cell-ny-first-timers-rate", "column": "Rate", "row": "ALL Candidates", "text": "73%" } ], "format": "table-cell-transcription", "locator": "FIRST TIMERS, ALL Candidates, Combined 2022", "normalization": "exact-cell-text", "span_id": "span-ny-first-timers-2022", "supports": "The combined first-time pass rate was 73%, so the complementary non-pass proportion used by the re-analysis is 27%." }

Work: em:dossier-source-work:sha256:51aa3262776f5da4a8ef12ab7f49366ef761951213e3a38f519a0a6b5d6534f1
Edition: em:dossier-edition:sha256:10bbf9ff899ba3ad12de154e0ab9ea7cc3f34c0f2263e79951e26539911bec5f
Edition digest: sha256:bbdcb28fb38b0d7d03ddb4e4bcfbcca170d2443491a180fee0978c7510cdb74f
Span: em:dossier-span:sha256:468763dc341321dd638ef9a08f6606ac5e02806a4086e4b0ba7bc05400c1d703
Span digest: sha256:e55680bc3d09d5c2968a8449c122db232a89c5f89b29dc9944b69c2b1eeeee11
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

Appendix for GPT-4 passes the Bar Exam

§4(d), Table 7

{ "cells": [ { "cell_id": "cell-katz-gpt4-mbe", "column": "GPT-4", "row": "MBE", "text": "157 points" }, { "cell_id": "cell-katz-gpt4-mee", "column": "GPT-4", "row": "MEE", "text": "84 points" }, { "cell_id": "cell-katz-gpt4-mpt", "column": "GPT-4", "row": "MPT", "text": "56 points" }, { "cell_id": "cell-katz-gpt4-overall", "column": "GPT-4", "row": "overall score", "text": "297 points" } ], "format": "table-cell-transcription", "locator": "\u00a74(d), Table 7", "normalization": "exact-cell-text", "span_id": "span-katz-vor-components", "supports": "The version-of-record component scores sum to 297." }

Work: em:dossier-source-work:sha256:880443d56eafdb224149d953f7de48e7550d53f814008551895b180ecb4b203b
Edition: em:dossier-edition:sha256:21621ede6a23bb3f98867c0918e5d9808d6dced5ca3f271a3a52e561810f2f7b
Edition digest: sha256:9d10cfedd531e62d96ea27bb78b00a9148e6b5d8135807836c5982e65a42416e
Span: em:dossier-span:sha256:293687df5cf4b74d839237414173205591aacf4850103e1621aece4f21fb763b
Span digest: sha256:e6c808d4b4dcd243210518497a0c8788d06ee91d8571c474c13514c2c8afe04b
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree · CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree

percentile_analysis_new.Rmd

introduction, journal p. 584

The paper successfully replicates the MBE score of 158, but highlights several methodological issues in the grading of the MPT + MEE components of the exam, which call into question the validity of the essay score (140).

Work: em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b
Edition: em:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667
Edition digest: sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5
Span: em:dossier-span:sha256:47ce609849ae1ff1224df62ce9b453707dd9b7a93cd4413822c84178eac7f853
Span digest: sha256:11db34f6066f861f338266e98c11852743b727734f62c1a773ae257bf219e609
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed

percentile_analysis_new.Rmd

lines 55-58

{ "cells": [ { "cell_id": "cell-martinez-july-mbe-85", "count": 2, "score": 85 }, { "cell_id": "cell-martinez-july-mbe-90", "count": 2, "score": 90 }, { "cell_id": "cell-martinez-july-mbe-95", "count": 5, "score": 95 }, { "cell_id": "cell-martinez-july-mbe-100", "count": 6, "score": 100 }, { "cell_id": "cell-martinez-july-mbe-105", "count": 13, "score": 105 }, { "cell_id": "cell-martinez-july-mbe-110", "count": 22, "score": 110 }, { "cell_id": "cell-martinez-july-mbe-115", "count": 33, "score": 115 }, { "cell_id": "cell-martinez-july-mbe-120", "count": 56, "score": 120 }, { "cell_id": "cell-martinez-july-mbe-125", "count": 73, "score": 125 }, { "cell_id": "cell-martinez-july-mbe-130", "count": 78, "score": 130 }, { "cell_id": "cell-martinez-july-mbe-135", "count": 104, "score": 135 }, { "cell_id": "cell-martinez-july-mbe-140", "count": 96, "score": 140 }, { "cell_id": "cell-martinez-july-mbe-145", "count": 101, "score": 145 }, { "cell_id": "cell-martinez-july-mbe-150", "count": 99, "score": 150 }, { "cell_id": "cell-martinez-july-mbe-155", "count": 99, "score": 155 }, { "cell_id": "cell-martinez-july-mbe-160", "count": 79, "score": 160 }, { "cell_id": "cell-martinez-july-mbe-165", "count": 64, "score": 165 }, { "cell_id": "cell-martinez-july-mbe-170", "count": 38, "score": 170 }, { "cell_id": "cell-martinez-july-mbe-175", "count": 22, "score": 175 }, { "cell_id": "cell-martinez-july-mbe-180", "count": 8, "score": 180 }, { "cell_id": "cell-martinez-july-mbe-185", "count": 2, "score": 185 } ], "code_lines": [ { "line_id": "line-martinez-july-distribution-55", "text": "July_distribution <- c(rep(85, 2), rep(90, 2), rep(95, 5), rep(100, 6), rep(105, 13)," }, { "line_id": "line-martinez-july-distribution-56", "text": " rep(110, 22), rep(115, 33), rep(120, 56), rep(125, 73), rep(130, 78)," }, { "line_id": "line-martinez-july-distribution-57", "text": " rep(135, 104), rep(140, 96), rep(145, 101), rep(150, 99), rep(155, 99)," }, { "line_id": "line-martinez-july-distribution-58", "text": " rep(160, 79), rep(165, 64), rep(170, 38), rep(175, 22), rep(180, 8), rep(185, 2))" } ], "format": "code-table-transcription", "locator": "lines 55-58", "normalization": "exact-code-text", "span_id": "span-martinez-script-july-mbe-distribution", "supports": "The executable analysis expands these 21 score-count cells to estimate the July MBE sample standard deviation." }

Work: em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b
Edition: em:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745
Edition digest: sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666
Span: em:dossier-span:sha256:c93f04ab5c92ae7955b75278cff1f0392210fc5462823dbe30cba42b5ffbab38
Span digest: sha256:9912e2d4f1e369b4f9529d27b6d1fef5936814f0c6151a9fe9f675af8c375e0c
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed

percentile_analysis_new.Rmd

lines 104-112

{ "format": "code-segment-transcription", "locator": "lines 104-112", "normalization": "exact-code-text", "segments": [ { "segment_id": "segment-martinez-code-mean", "text": "mean <- 287.6" }, { "segment_id": "segment-martinez-code-quantile", "text": "quantile_value <- 266" }, { "segment_id": "segment-martinez-code-percentile", "text": "percentile <- 0.27 # 24%" }, { "segment_id": "segment-martinez-code-z-score", "text": "z_score <- qnorm(percentile)" }, { "segment_id": "segment-martinez-code-ube-sd", "text": "sd_UBE <- (quantile_value - mean) / z_score" } ], "span_id": "span-martinez-script-ube-sd", "supports": "The script derives UBE standard deviation from mean 287.6, cutoff 266, and non-pass proportion 0.27." }

Work: em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b
Edition: em:dossier-edition:sha256:b135493e681d789459bf386b90f79308cc68da4559b410ad6e818779a81f2745
Edition digest: sha256:c90a873989a59d231475a13a92e3ad1d0d3276514ad54e4e5df695cd35e48666
Span: em:dossier-span:sha256:7c297fee53ebcae4ddc2b4bd2301d9742b570914c364b3993890c92c3ab42d83
Span digest: sha256:29d389334aa2b12045b1d9febb630b66d80d832615d0764534e4e0a2ac49b55b
Retrieval: No external retrieval record; repository audit span
License: CC BY 4.0; no file license confirmed; no node or file license confirmed · CC BY 4.0; no file license confirmed; no node or file license confirmed

The Testing Column: Why Are February Bar Exam Pass Rates Lower than July Pass Rates?

paragraph beginning 'The numbers from 2019'

The numbers from 2019, the last prepandemic year, are typical: all likely first-time test takers earned an average MBE score of 143.8. Likely repeaters, however, earned an average MBE score of 132.4.

Work: em:dossier-source-work:sha256:802fad21dc2e69f5baff90d345e036b898bf958351af8eae39f346d23e005a12
Edition: em:dossier-edition:sha256:852a2ad23d9f6f760f0f21c4147eb6dd1b3c936ea9704e3dc3f743de193e4e49
Edition digest: sha256:460f8d22d10526ee2a92e1fb4045e1cabacb0dc4a668e142a91a30592ccf0ed2
Span: em:dossier-span:sha256:6f0efeaf8fa67729f3f75793e1b64ce9961bb3463e5acc9d9c4a1d5aeb588520
Span digest: sha256:1452bea6f3158b2b58d90800d02a6cf2348935e6a194e07ab22efc88c5f26302
Retrieval: No external retrieval record; repository audit span
License: no open license confirmed · no open license confirmed

Verify every number

Complete count ledgers

Every displayed total is a view over these typed members; no total is maintained as marketing copy.

1 · Historical Model Run

root-historical-simulated-ube-run Historical simulated-UBE model-performance root
Member
root-historical-simulated-ube-run
Object type
lineages
Object key
lineage-model-performance-root
Object ID
em:dossier-lineage:sha256:c8b81e09b0245e5c5302dd89be2d627f7658d7ab0d5de6d56adef3df0f65e145
Status
known
Dependence
data, method, model, social
Basis span
em:dossier-span:sha256:ee862ec363b34142a0bebb11247959c3457a8982de69653364d7d36f10059703 sha256:57920d8649f509df54908d08dfaefb9860e1afc005095c6cd4cb051344dff044
Basis edition
em:dossier-edition:sha256:21621ede6a23bb3f98867c0918e5d9808d6dced5ca3f271a3a52e561810f2f7b sha256:9d10cfedd531e62d96ea27bb78b00a9148e6b5d8135807836c5982e65a42416e
Basis work
em:dossier-source-work:sha256:880443d56eafdb224149d953f7de48e7550d53f814008551895b180ecb4b203b · CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree

3 · Official Administration Sensitivities

root-illinois-february-2018 Illinois February 2018 comparison root
Member
root-illinois-february-2018
Object type
lineages
Object key
lineage-illinois-comparison-root
Object ID
em:dossier-lineage:sha256:a5b9820aa7d6c9062b89944f259392c039fbe5285b626d4f71d54dfd44f51a20
Status
known
Dependence
data, source
Basis span
em:dossier-span:sha256:56a011a09e4e7dd05afbc0309c94d4dcfa1ecf1d6fd3df6a6a6f65204c7ccf03 sha256:b3b8f789db7571a4fd8ed73e5380f2847aae920172722b9f6f51eec96c62e9ce
Basis edition
em:dossier-edition:sha256:9be0e724fc35f74c96e635c6263f8f44ef804c3406c34b268bb710b24228a368 sha256:44de7bd89cd4aeacee7b38d7077ff659e3baf552cb9b295401ec9b7229c04964
Basis work
em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd · no open license confirmed
root-illinois-february-2019 Illinois February 2019 comparison root
Member
root-illinois-february-2019
Object type
lineages
Object key
lineage-illinois-comparison-root
Object ID
em:dossier-lineage:sha256:a5b9820aa7d6c9062b89944f259392c039fbe5285b626d4f71d54dfd44f51a20
Status
known
Dependence
data, source
Basis span
em:dossier-span:sha256:1e2736835309f364d7f8553b6e26a7c9802dce86c720c84fa2097f7e3a755ca3 sha256:29f7ab38f7ff54e4576aa5b3c6fd310debd0520513f68fc3e21744af073923f8
Basis edition
em:dossier-edition:sha256:2c2badc4c75c58ca49007bda1f0c1fcad4b301ecd060bc905bd8a741acd80925 sha256:91185bd6263d73e5976b5b00f28fb9b62301be0bb0347b1d0a308c295f6a6151
Basis work
em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd · no open license confirmed
root-illinois-july-2018 Illinois July 2018 comparison root
Member
root-illinois-july-2018
Object type
lineages
Object key
lineage-illinois-comparison-root
Object ID
em:dossier-lineage:sha256:a5b9820aa7d6c9062b89944f259392c039fbe5285b626d4f71d54dfd44f51a20
Status
known
Dependence
data, source
Basis span
em:dossier-span:sha256:3ec4b97842a7b34ea0a2857a50da7f6a02eb1556351c7ec727b2cccd2743637c sha256:fe399dcc350821ac8ab546593931f1e34b24d811cad9062126e487a53783a783
Basis edition
em:dossier-edition:sha256:2b720aa3b7368249b079f4bc7d3de383efc590510bc33e1ed88db133e9293237 sha256:a9aa31983924ffec8d0e713ad29fc16a0e466557a57eba5061577afc4698bae6
Basis work
em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd · no open license confirmed

7 · Rooted Evidence Units

root-model-performance Historical model-performance root
Member
root-model-performance
Object type
lineages
Object key
lineage-model-performance-root
Object ID
em:dossier-lineage:sha256:c8b81e09b0245e5c5302dd89be2d627f7658d7ab0d5de6d56adef3df0f65e145
Status
known
Dependence
data, method, model, social
Basis span
em:dossier-span:sha256:ee862ec363b34142a0bebb11247959c3457a8982de69653364d7d36f10059703 sha256:57920d8649f509df54908d08dfaefb9860e1afc005095c6cd4cb051344dff044
Basis edition
em:dossier-edition:sha256:21621ede6a23bb3f98867c0918e5d9808d6dced5ca3f271a3a52e561810f2f7b sha256:9d10cfedd531e62d96ea27bb78b00a9148e6b5d8135807836c5982e65a42416e
Basis work
em:dossier-source-work:sha256:880443d56eafdb224149d953f7de48e7550d53f814008551895b180ecb4b203b · CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree
root-martinez-reanalysis Martinez modeled re-analysis root
Member
root-martinez-reanalysis
Object type
lineages
Object key
lineage-martinez-analysis-root
Object ID
em:dossier-lineage:sha256:e6057f0dbfa37d0264e471e67a3941297a56467e67cad715d0eb37490d320cdd
Status
known
Dependence
data, method, model, source
Basis span
em:dossier-span:sha256:e4d9982c357259d2e11c9d6e647569ca0338134240924a766792fbba54de7502 sha256:94dbb34cb0b30aefe6ad46a54298be28df4bf1014c3a0d389f63e4615b3dc9bf
Basis edition
em:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667 sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5
Basis work
em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed
root-illinois-feb-2018 Illinois February 2018 root
Member
root-illinois-feb-2018
Object type
lineages
Object key
lineage-illinois-comparison-root
Object ID
em:dossier-lineage:sha256:a5b9820aa7d6c9062b89944f259392c039fbe5285b626d4f71d54dfd44f51a20
Status
known
Dependence
data, source
Basis span
em:dossier-span:sha256:56a011a09e4e7dd05afbc0309c94d4dcfa1ecf1d6fd3df6a6a6f65204c7ccf03 sha256:b3b8f789db7571a4fd8ed73e5380f2847aae920172722b9f6f51eec96c62e9ce
Basis edition
em:dossier-edition:sha256:9be0e724fc35f74c96e635c6263f8f44ef804c3406c34b268bb710b24228a368 sha256:44de7bd89cd4aeacee7b38d7077ff659e3baf552cb9b295401ec9b7229c04964
Basis work
em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd · no open license confirmed
root-illinois-feb-2019 Illinois February 2019 root
Member
root-illinois-feb-2019
Object type
lineages
Object key
lineage-illinois-comparison-root
Object ID
em:dossier-lineage:sha256:a5b9820aa7d6c9062b89944f259392c039fbe5285b626d4f71d54dfd44f51a20
Status
known
Dependence
data, source
Basis span
em:dossier-span:sha256:1e2736835309f364d7f8553b6e26a7c9802dce86c720c84fa2097f7e3a755ca3 sha256:29f7ab38f7ff54e4576aa5b3c6fd310debd0520513f68fc3e21744af073923f8
Basis edition
em:dossier-edition:sha256:2c2badc4c75c58ca49007bda1f0c1fcad4b301ecd060bc905bd8a741acd80925 sha256:91185bd6263d73e5976b5b00f28fb9b62301be0bb0347b1d0a308c295f6a6151
Basis work
em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd · no open license confirmed
root-illinois-jul-2018 Illinois July 2018 root
Member
root-illinois-jul-2018
Object type
lineages
Object key
lineage-illinois-comparison-root
Object ID
em:dossier-lineage:sha256:a5b9820aa7d6c9062b89944f259392c039fbe5285b626d4f71d54dfd44f51a20
Status
known
Dependence
data, source
Basis span
em:dossier-span:sha256:3ec4b97842a7b34ea0a2857a50da7f6a02eb1556351c7ec727b2cccd2743637c sha256:fe399dcc350821ac8ab546593931f1e34b24d811cad9062126e487a53783a783
Basis edition
em:dossier-edition:sha256:2b720aa3b7368249b079f4bc7d3de383efc590510bc33e1ed88db133e9293237 sha256:a9aa31983924ffec8d0e713ad29fc16a0e466557a57eba5061577afc4698bae6
Basis work
em:dossier-source-work:sha256:a2e64d0c97109c831bb2facc8c180da8179ed37526502ea73dbe3016fdf003dd · no open license confirmed
root-ncbe-aggregates NCBE scoring and population aggregate root
Member
root-ncbe-aggregates
Object type
lineages
Object key
lineage-ncbe-comparison-root
Object ID
em:dossier-lineage:sha256:a674ad6aa3453c54eb437d844fd6070006c25bbb05cb6463bfffd2eee9e24eae
Status
known
Dependence
data, source
Basis span
em:dossier-span:sha256:b00f1cab5f375334f432b9de6ffb480cebae7ee1557ad424feb6a6223ae1cf25 sha256:0f6d015b519414ebec9cd4f12bbf7dea62dc9f77fb21378d73690b451d5f0f1e
Basis edition
em:dossier-edition:sha256:3b7b0ad7942bb25d248c4fffe8e6bcfa1b806e7f695c9cda6ffd170359cc1ba6 sha256:ab04a05f122d9dff51b76f2e9be9f78c0bdcf25f58fc62047cc4c5abd040346c
Basis work
em:dossier-source-work:sha256:324982bb2b90295b84f9e309ab84ca9a46cd0f1be9e9e3079a18bcc20bf58e63 · no open license confirmed
root-new-york-pass-rates New York first-time pass-rate root
Member
root-new-york-pass-rates
Object type
lineages
Object key
lineage-new-york-pass-rate-root
Object ID
em:dossier-lineage:sha256:b426c4edd6b881b995272c355d5dc869ffe4153c0bcbe662e7ffef1a5267e48b
Status
known
Dependence
data, source
Basis span
em:dossier-span:sha256:468763dc341321dd638ef9a08f6606ac5e02806a4086e4b0ba7bc05400c1d703 sha256:e55680bc3d09d5c2968a8449c122db232a89c5f89b29dc9944b69c2b1eeeee11
Basis edition
em:dossier-edition:sha256:10bbf9ff899ba3ad12de154e0ab9ea7cc3f34c0f2263e79951e26539911bec5f sha256:bbdcb28fb38b0d7d03ddb4e4bcfbcca170d2443491a180fee0978c7510cdb74f
Basis work
em:dossier-source-work:sha256:51aa3262776f5da4a8ef12ab7f49366ef761951213e3a38f519a0a6b5d6534f1 · no open license confirmed

3 · Unresolved Boundaries

boundary-launch-comparator Launch comparison distribution is unspecified
Member
boundary-launch-comparator
Object type
assertions
Object key
assertion-claim-launch-comparison-unspecified
Object ID
em:dossier-assertion:sha256:e5b09f98417934e590aa916b7e5c657dc707fb55ab33c81d87a6a724d48fc159
Basis span
em:dossier-span:sha256:ca6ba91cbc68c2a3aa343fc059aa3058d27baa364e998d9c05393578c6045d5c sha256:53c5c3cd65f0c06ae12ec339f9a8f503b977ebaf9c1cd8723d9caecfa42bcdcb
Basis edition
em:dossier-edition:sha256:21621ede6a23bb3f98867c0918e5d9808d6dced5ca3f271a3a52e561810f2f7b sha256:9d10cfedd531e62d96ea27bb78b00a9148e6b5d8135807836c5982e65a42416e
Basis work
em:dossier-source-work:sha256:880443d56eafdb224149d953f7de48e7550d53f814008551895b180ecb4b203b · CC BY 4.0; CC BY 4.0 at the article/file level; collection-level license null; no open license confirmed; no repository license at the pinned tree
boundary-45-48 Martinez 45th-versus-48th internal discrepancy
Member
boundary-45-48
Object type
assertions
Object key
assertion-claim-martinez-passers-conflict
Object ID
em:dossier-assertion:sha256:35a975b45cb9ebe6fed26d97fde109f534f7ce69613026271c16226ebf3c1182
Basis span
em:dossier-span:sha256:e4d9982c357259d2e11c9d6e647569ca0338134240924a766792fbba54de7502 sha256:94dbb34cb0b30aefe6ad46a54298be28df4bf1014c3a0d389f63e4615b3dc9bf
Basis edition
em:dossier-edition:sha256:65219a4731cb2e79f58629db4793e659a48da74aa74c3a3db5049ce260701667 sha256:c74c66e3adc8a45af5bac9e338c11d927dd2d95c8386b2d4bb52ef2591c7a0a5
Basis work
em:dossier-source-work:sha256:3e91b5cabaf08b648f9e9e505d7fa5e9519cca7becf3ec753b1777075e4d512b · CC BY 4.0; no file license confirmed; no node or file license confirmed
boundary-lawyer-quality No practicing-lawyer comparison
Member
boundary-lawyer-quality
Object type
assertions
Object key
assertion-claim-no-lawyer-rank
Object ID
em:dossier-assertion:sha256:3c3793e084854e911d7a2aece20cf4a7f9f430d3f71a36e87cee09ed9228a514
Basis span
em:dossier-span:sha256:c574b1bd1adfdbb6b446b321eb656c49dc7a710ed609089c52069dd4da29d296 sha256:61393f3895d80b5bb5e4e4240ff6c4b8bb9246f22b58ac1a766643c63887e95a
Basis edition
em:dossier-edition:sha256:859fa96f390685d21c4db9633a6d9c7fec7abe76cdf8a1f8c0c9c82501b193eb sha256:1703b719a1fad06284c9cde691496e2ee6cc9371ab7808d8abfc50ab3ed0c2f9
Basis work
em:dossier-source-work:sha256:2c6c231a62e11f961467d990d3bd0ad2d4d4568fcd39891c16e6fe4622e9448b · arXiv non-exclusive distribution license; no Creative Commons license confirmed

Five-line lexicon

The units are different on purpose

Comparison population: The people or scores against which a percentile is calculated.
Administration sensitivity: How the same score ranks under a different exam date and population.
Historical model run: One past evaluation setup, not a claim about a current product.
Modeled re-analysis: A calculation using disclosed assumptions and reused source values.
Boundary: A missing comparator, conflict, or scope limit that remains visible.

The units count different things

What the selected record actually supports

Typed dependence

GPT-4 Technical Report

GPT-4 Technical Report

Appendix for GPT-4 passes the Bar Exam

Appendix for GPT-4 passes the Bar Exam

percentile_analysis_new.Rmd

Appendix for GPT-4 passes the Bar Exam

GPT-4 Technical Report

GPT-4 Technical Report

Appendix for GPT-4 passes the Bar Exam

Typed dependence

Appendix for GPT-4 passes the Bar Exam

Appendix for GPT-4 passes the Bar Exam

GPT-4 Technical Report

GPT-4 Technical Report

Appendix for GPT-4 passes the Bar Exam

Appendix for GPT-4 passes the Bar Exam

percentile_analysis_new.Rmd

Appendix for GPT-4 passes the Bar Exam

GPT-4 Technical Report

GPT-4 Technical Report

Typed dependence

Illinois February 2018 Bar Examination Percentile Equivalents

Illinois February 2018 Bar Examination Percentile Equivalents

Illinois February 2018 Bar Examination Percentile Equivalents

Typed dependence

Illinois February 2018 Bar Examination Percentile Equivalents

Illinois February 2018 Bar Examination Percentile Equivalents

Illinois February 2018 Bar Examination Percentile Equivalents

Typed dependence

percentile_analysis_new.Rmd

percentile_analysis_new.Rmd

percentile_analysis_new.Rmd

First-Time Exam Takers and Repeaters in 2022

New York Bar Exam 2022 Statistics

Illinois February 2018 Bar Examination Percentile Equivalents

Illinois February 2018 Bar Examination Percentile Equivalents

Illinois February 2018 Bar Examination Percentile Equivalents

Appendix for GPT-4 passes the Bar Exam

First-Time Exam Takers and Repeaters in 2022

First-Time Exam Takers and Repeaters in 2022

Appendix for GPT-4 passes the Bar Exam

percentile_analysis_new.Rmd

percentile_analysis_new.Rmd

percentile_analysis_new.Rmd

The Testing Column: Why Are February Bar Exam Pass Rates Lower than July Pass Rates?

Typed dependence

percentile_analysis_new.Rmd

percentile_analysis_new.Rmd

percentile_analysis_new.Rmd

percentile_analysis_new.Rmd

Illinois February 2018 Bar Examination Percentile Equivalents

Illinois February 2018 Bar Examination Percentile Equivalents

Illinois February 2018 Bar Examination Percentile Equivalents

Appendix for GPT-4 passes the Bar Exam

First-Time Exam Takers and Repeaters in 2022

First-Time Exam Takers and Repeaters in 2022

First-Time Exam Takers and Repeaters in 2022

New York Bar Exam 2022 Statistics

Appendix for GPT-4 passes the Bar Exam

percentile_analysis_new.Rmd

percentile_analysis_new.Rmd

percentile_analysis_new.Rmd

The Testing Column: Why Are February Bar Exam Pass Rates Lower than July Pass Rates?

Complete count ledgers

The units are different on purpose

Reproduce this projection