Welcome to OCELoT!
From June 22nd to June 29th, OCELoT will be used to collect submissions to the Shared Task: Machine Translation of News which is part of the EMNLP 2020 Fifth Conference on Machine Translation (WMT20), replacing the previously used matrix which had grown stale over time. You can read more about this year's shared task and changes compared to previous years in the competition updates section. We're looking forward to your participation in WMT20!
From July 10th to July 17th, OCELoT will collect submissions to the Shared Task: Machine Translation Robustness.
Competition updates
- 7/17/2020: deadline for the WMT20 robustness task has been extended until July 20th, anywhere on Earth.
- 7/16/2020: submission is (finally) now open for the WMT20 robustness task, enjoy!
- 7/09/2020: finalising list of primary submissions for the news task. Also preparing for submissions to the robustness task, to run from July 10—17.
- 6/25/2020: changed SacreBLEU computation for Khmer (
'char-based'tokeniser) target language. Also added chrF computation to give you more data points for your primary submission selection. Submissions remain ordered by decreasing SacreBLEU score. - 6/24/2020: changed SacreBLEU computation for Chinese (
'zh'tokeniser) and Japanese ('char-based'tokeniser) target languages. Score-1.0used for any submissions with invalid SGML files. If your system shows as-1.0, please check your SGML file and avoid any<, >, &or other HTML entities within your<seg/>segments. - 6/23/2020: submission is now open and competition updates have been added to the OCELoT website. 48 teams have signed up so far.
- 6/22/2020: team registration is online and newstest2020 test set source data has been released.
Competition rules
The following rules apply to this year's shared task:
- We only accept submissions from verified teams — you will have to contact us to verify your team account;
- Each team may upload up to 7 submissions per language pair, without any time contraints between such uploads;
- Each team may choose exactly one primary submission to be included in the human evaluation campaign;
- If you fail to designate your primary submission, your highest-scoring submission will be the default;
- We request that you do not create multiple teams to game these submission rules — we trust and thank you!