Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
dev:translators:framework [2011/05/23 16:42] โ [Example Translator] ajlyon | dev:translators:framework [2017/11/18 16:20] (current) โ Add legacy notice adamsmith | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | **While the Translator Framework still works, new translators using the Framework are no longer accepted, and we are migrating existing translators away from the format.** | ||
+ | |||
+ | **This page exists as legacy documentation only.** | ||
+ | |||
====== Translator Framework ====== | ====== Translator Framework ====== | ||
The translator framework is a way to build web translators that lets translator authors avoid most of the boilerplate that usually is required for new translators, | The translator framework is a way to build web translators that lets translator authors avoid most of the boilerplate that usually is required for new translators, | ||
- | The framework was written and contributed by Erik Hetzner and is licensed under the GPLv3+. It currently resides at http://e6h.org/~egh/hg/ | + | The framework was written and contributed by Erik Hetzner and is licensed under the AGPLv3+. It currently resides at https://gitlab.com/ |
- | To use the framework, simply insert the framework code at the beginning of your translator, after the translator information block (JSON header). If you are using [[dev/ | + | To use the framework, simply insert the framework code at the beginning of your translator, after the translator information block (JSON header). If you are using [[dev/ |
You'll start writing beneath the line that reads: | You'll start writing beneath the line that reads: | ||
''/ | ''/ | ||
- | **Note: The most recent | + | **Note:** The latest |
====Example Translator==== | ====Example Translator==== | ||
From EurasiaNet.js: | From EurasiaNet.js: | ||
Line 55: | Line 58: | ||
=== FW.Scraper === | === FW.Scraper === | ||
- | * Required keys: '' | + | * Required keys: '' |
- | * Optional keys: '' | + | * Optional keys: '' |
=== FW.MultiScraper === | === FW.MultiScraper === | ||
- | * Required keys: '' | + | * Required keys: '' |
- | * Optional keys: '' | + | * Optional keys: '' |
- | The '' | + | The key '' |
+ | <code javascript> | ||
+ | choices : { | ||
+ | | ||
+ | | ||
+ | } | ||
+ | </ | ||
== Pre-processing == | == Pre-processing == | ||
Line 80: | Line 89: | ||
Here the option is used to guarantee that the multiple item page has links to the BibTeX files that the translator uses. | Here the option is used to guarantee that the multiple item page has links to the BibTeX files that the translator uses. | ||
- | == Delegation | + | === Attachments === |
- | It is possible | + | To add attachments |
+ | The keys '' | ||
<code javascript> | <code javascript> | ||
- | itemTrans | + | |
- | | + | { url : FW.Url(), |
+ | title : FW.Url().match(/ | ||
+ | type : " | ||
+ | | ||
+ | | ||
+ | key(' | ||
+ | match(/[0-9]+$/). | ||
+ | prepend(" | ||
+ | type : " | ||
+ | ] | ||
</ | </ | ||
- | |||
- | This delegation method can only be used with a '' | ||
=== Post-Processing with Hooks === | === Post-Processing with Hooks === | ||
Line 107: | Line 124: | ||
In the example above, the scraper had been written to save two potential date fields, one as " | In the example above, the scraper had been written to save two potential date fields, one as " | ||
===== Functions ===== | ===== Functions ===== | ||
- | To use the framework, just chain together functions from the list below until you get the desired output. | + | To use the framework, just chain together functions from the list below until you get the desired output. Note that JavaScript functions not in this list will not work within the scrapers. |
=== Main functions === | === Main functions === | ||
* '' | * '' | ||
Line 118: | Line 135: | ||
* '' | * '' | ||
* '' | * '' | ||
- | * '' | + | * '' |
* '' | * '' | ||
* '' | * '' | ||
- | * '' | + | * '' |
* '' | * '' | ||
* '' | * '' | ||
* '' | * '' | ||
- | * '' | + | * '' |
- | * '' | + | * '' |
- | === Zotero functions | + | * '' |
- | | + | |
- | | + | === Putting things together |
+ | '' | ||
+ | |||
+ | You can also call a method on this object, e.g.: | ||
+ | |||
+ | FW.Xpath("// | ||
+ | |||
+ | This modifies the object to include a filter that splits the | ||
+ | text on /,/. You can chain these together: | ||
+ | |||
+ | FW.Xpath("// | ||
+ | |||
+ | This will split the text returned by the XPath into an array and call | ||
+ | the filters (trim, then cleanAuthor) on each member of the array. This | ||
+ | way we can add multiple creators to the item. | ||
+ | |||
+ | If you want to add an arbitrary filter, this should work: | ||
+ | |||
+ | FW.Xpath("// | ||
+ | |||
+ | ===== Templates ===== | ||
+ | Just paste the following templates into your translator, fill in the appropriate fields, and delete the unnecessary fields. | ||
+ | ==== Boilerplate ==== | ||
+ | Required for every framework-derived translator. | ||
+ | <code javascript> | ||
+ | function detectWeb(doc, | ||
+ | function doWeb(doc, url) { return FW.doWeb(doc, | ||
+ | </ | ||
+ | ==== FW.MultiScraper ==== | ||
+ | <code javascript> | ||
+ | FW.MultiScraper({ | ||
+ | itemType | ||
+ | detect | ||
+ | choices | ||
+ | | ||
+ | urls : | ||
+ | attachments : // optional | ||
+ | } | ||
+ | }); | ||
+ | </ | ||
+ | ==== FW.Scraper ==== | ||
+ | For possible values of '' | ||
+ | <code javascript> | ||
+ | FW.Scraper({ | ||
+ | itemType | ||
+ | detect | ||
+ | attachments | ||
+ | creators | ||
+ | hooks : { | ||
+ | " | ||
+ | } | ||
+ | // All possible fields | ||
+ | abstractNote : , | ||
+ | applicationNumber : , | ||
+ | archive : , | ||
+ | archiveLocation : , | ||
+ | artworkMedium : , | ||
+ | artworkSize : , | ||
+ | assignee : , | ||
+ | audioFileType : , | ||
+ | audioRecordingType : , | ||
+ | billNumber : , | ||
+ | blogTitle : , | ||
+ | bookTitle : , | ||
+ | callNumber : , | ||
+ | caseName : , | ||
+ | code : , | ||
+ | codeNumber : , | ||
+ | codePages : , | ||
+ | codeVolume : , | ||
+ | committee : , | ||
+ | company : , | ||
+ | conferenceName : , | ||
+ | country : , | ||
+ | court : , | ||
+ | date : , | ||
+ | dateDecided : , | ||
+ | dateEnacted : , | ||
+ | dictionaryTitle : , | ||
+ | distributor : , | ||
+ | docketNumber : , | ||
+ | documentNumber : , | ||
+ | DOI : , | ||
+ | edition : , | ||
+ | encyclopediaTitle : , | ||
+ | episodeNumber : , | ||
+ | extra : , | ||
+ | filingDate : , | ||
+ | firstPage : , | ||
+ | forumTitle : , | ||
+ | genre : , | ||
+ | history : , | ||
+ | institution : , | ||
+ | interviewMedium : , | ||
+ | ISBN : , | ||
+ | ISSN : , | ||
+ | issue : , | ||
+ | issueDate : , | ||
+ | issuingAuthority : , | ||
+ | journalAbbreviation : , | ||
+ | label : , | ||
+ | language : , | ||
+ | legalStatus : , | ||
+ | legislativeBody : , | ||
+ | letterType : , | ||
+ | libraryCatalog : , | ||
+ | manuscriptType : , | ||
+ | mapType : , | ||
+ | medium : , | ||
+ | meetingName : , | ||
+ | nameOfAct : , | ||
+ | network : , | ||
+ | number : , | ||
+ | numberOfVolumes : , | ||
+ | numPages : , | ||
+ | pages : , | ||
+ | patentNumber : , | ||
+ | place : , | ||
+ | postType : , | ||
+ | presentationType : , | ||
+ | priorityNumbers : , | ||
+ | proceedingsTitle : , | ||
+ | programTitle : , | ||
+ | programmingLanguage : , | ||
+ | publicLawNumber : , | ||
+ | publicationTitle : , | ||
+ | publisher : , | ||
+ | references : , | ||
+ | reportNumber : , | ||
+ | reportType : , | ||
+ | reporter : , | ||
+ | reporterVolume : , | ||
+ | rights : , | ||
+ | runningTime : , | ||
+ | scale : , | ||
+ | section : , | ||
+ | series : , | ||
+ | seriesNumber : , | ||
+ | seriesText : , | ||
+ | seriesTitle : , | ||
+ | session : , | ||
+ | shortTitle : , | ||
+ | studio : , | ||
+ | subject : , | ||
+ | system : , | ||
+ | thesisType : , | ||
+ | title : , | ||
+ | type : , | ||
+ | university : , | ||
+ | url : , | ||
+ | version : , | ||
+ | videoRecordingType : , | ||
+ | volume : , | ||
+ | websiteTitle : , | ||
+ | websiteType : | ||
+ | }); | ||
+ | </ |