Skip to content

Commit f780a1d

Browse files
kyuridenamidakyuridenamida
authored
Reinforce unit tests & Support C# & error type on documentation page (kyuridenamida#157)
* Reinforce unit tests & Support C# & error type on documentation page * judge_type -> judge_method * fix style issue * Bugfix around zero division & Better UX for decimal number judge * fix style * fix style * revert wrong change
1 parent d48cda2 commit f780a1d

File tree

17 files changed

+344
-130
lines changed

17 files changed

+344
-130
lines changed

‎README.md

Lines changed: 25 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@ Python 3.5 以降で動作する [AtCoder](http://atcoder.jp/) からサンプ
1010
- AtCoderへのログイン，入出力例データなどの抽出
1111
- 枝刈り探索による高精度・高速な入力フォーマット解析 (ARC、ABC、AGCについては約9割ほど)
1212
- 問題文中に含まれるMOD値やYES/NO文字列等の定数値抽出
13+
- サンプルのローカルテスト機能
14+
- 誤差ジャッジに対応 by [@chaemon](https://github.com/chaemon/)
1315
- コード提出機能
1416
- 入力フォーマット解析結果や抽出した定数値を用いたテンプレートからのコード自動生成(以下の表に記載されている言語をサポートしています)
1517
- カスタムテンプレートに対応
@@ -23,6 +25,7 @@ Python 3.5 以降で動作する [AtCoder](http://atcoder.jp/) からサンプ
2325
|Python3|[@kmyk](https://github.com/kmyk/) (generator, template)|[@penpenpng](https://github.com/penpenpng/) (generator)|
2426
|D|[@penpenpng](https://github.com/penpenpng/) (generator, template)||
2527
|Nim|[@chaemon](https://github.com/chaemon/) (generator, template)||
28+
|C#|[@chaemon](https://github.com/chaemon/) (generator, template)||
2629

2730
## Demo
2831
<img src="https://user-images.githubusercontent.com/233559/52807100-f6e2d300-30cd-11e9-8906-82b9f9b2dff7.gif" width=70%>
@@ -90,7 +93,7 @@ optional arguments:
9093
--workspace WORKSPACE
9194
Path to workspace's root directory. This script will create files in {WORKSPACE}/{contest_name}/{alphabet}/ e.g. ./your-workspace/arc001/A/
9295
[Default] /home/kyuridenamida/atcoder-workspace
93-
--lang LANG Programming language of your template code, cpp or java or rust or python or nim or d.
96+
--lang LANG Programming language of your template code, cpp or java or rust or python or nim or d or cs.
9497
[Default] cpp
9598
--template TEMPLATE File path to your template code
9699
[Default (C++)] /atcodertools/tools/templates/default_template.cpp
@@ -99,6 +102,7 @@ optional arguments:
99102
[Default (Python3)] /atcodertools/tools/templates/default_template.py
100103
[Default (NIM)] /atcodertools/tools/templates/default_template.nim
101104
[Default (D)] /atcodertools/tools/templates/default_template.d
105+
[Default (C#)] /atcodertools/tools/templates/default_template.cs
102106
--parallel Prepare problem directories asynchronously using multi processors.
103107
--save-no-session-cache
104108
Save no session cache to avoid security risk
@@ -110,11 +114,12 @@ optional arguments:
110114
### test の詳細
111115

112116
```
113-
usage: atcoder-tools test [-h] [--exec EXEC]
114-
[--num NUM]
115-
[--dir DIR]
116-
[--timeout TIMEOUT]
117-
[--knock-out]
117+
usage: atcoder-tools test [-h] [--exec EXEC] [--num NUM]
118+
[--dir DIR] [--timeout TIMEOUT]
119+
[--knock-out]
120+
[--skip-almost-ac-feedback]
121+
[--judge-type JUDGE_TYPE]
122+
[--error-value ERROR_VALUE]
118123
119124
optional arguments:
120125
-h, --help show this help message and exit
@@ -126,19 +131,22 @@ optional arguments:
126131
--knock-out, -k Stop execution immediately after any example's failure [Default] False
127132
--skip-almost-ac-feedback, -s
128133
Hide inputs and expected/actual outputs if result is correct and there are error outputs [Default] False,
134+
--judge-type JUDGE_TYPE, -j JUDGE_TYPE
135+
error type must be one of [normal, absolute, relative, absolute_or_relative]
136+
--error-value ERROR_VALUE, -v ERROR_VALUE
137+
error value for decimal number judge: [Default] 0.000000001
129138
```
130139

131140

132141
### submit の詳細
133142

134143
```
135-
usage: atcoder-tools submit [-h] [--exec EXEC]
136-
[--dir DIR]
137-
[--timeout TIMEOUT]
138-
[--code CODE]
139-
[--force]
140-
[--save-no-session-cache]
141-
[--unlock-safety]
144+
usage: atcoder-tools submit [-h] [--exec EXEC] [--dir DIR]
145+
[--timeout TIMEOUT] [--code CODE]
146+
[--force] [--save-no-session-cache]
147+
[--unlock-safety]
148+
[--judge-type JUDGE_TYPE]
149+
[--error-value ERROR_VALUE]
142150
143151
optional arguments:
144152
-h, --help show this help message and exit
@@ -151,7 +159,10 @@ optional arguments:
151159
--save-no-session-cache
152160
Save no session cache to avoid security risk
153161
--unlock-safety, -u By default, this script only submits the first code per problem. However, you can remove the safety by this option in order to submit codes twice or more.
154-
162+
--judge-type JUDGE_TYPE, -j JUDGE_TYPE
163+
error type must be one of [normal, absolute, relative, absolute_or_relative]
164+
--error-value ERROR_VALUE, -v ERROR_VALUE
165+
error value for decimal number judge: [Default] 1e-09
155166
```
156167

157168
### codegen の詳細

‎atcodertools/common/judgetype.py

Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
#!/usr/bin/python3
22
# -*- coding: utf-8 -*-
3-
3+
from abc import ABCMeta, abstractmethod
44
from enum import Enum
55

66

@@ -16,10 +16,15 @@ class ErrorType(Enum):
1616
AbsoluteOrRelative = "absolute_or_relative"
1717

1818

19-
class Judge:
19+
class Judge(metaclass=ABCMeta):
20+
@abstractmethod
2021
def verify(self, output, expected):
2122
pass
2223

24+
@abstractmethod
25+
def to_dict(self):
26+
pass
27+
2328

2429
class NormalJudge(Judge):
2530
def __init__(self):
@@ -48,20 +53,25 @@ def __init__(self,
4853
self.error_type = error_type
4954
self.diff = diff
5055

51-
def __verify_sub(self, output, expected: float) -> bool:
56+
def _verify_sub(self, output: float, expected: float) -> bool:
5257
if self.error_type in [ErrorType.Absolute, ErrorType.AbsoluteOrRelative] and abs(expected - output) <= self.diff:
5358
return True
54-
if self.error_type in [ErrorType.Relative, ErrorType.AbsoluteOrRelative] and abs((expected - output) / expected) <= self.diff:
59+
if self.error_type in [ErrorType.Relative, ErrorType.AbsoluteOrRelative] and self._calc_absolute(output, expected):
5560
return True
5661
return False
5762

63+
def _calc_absolute(self, output: float, expected: float) -> bool:
64+
if expected == 0:
65+
return expected == output
66+
return abs((expected - output) / expected) <= self.diff
67+
5868
def verify(self, output, expected) -> bool:
5969
output = output.strip().split()
6070
expected = expected.strip().split()
6171
if len(output) != len(expected):
6272
return False
6373
for i in range(0, len(output)):
64-
if not self.__verify_sub(float(output[i]), float(expected[i])):
74+
if not self._verify_sub(float(output[i]), float(expected[i])):
6575
return False
6676
return True
6777

‎atcodertools/constprediction/constants_prediction.py

Lines changed: 14 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,10 @@
33

44
from bs4 import BeautifulSoup
55

6-
from atcodertools.constprediction.models.problem_constant_set import ProblemConstantSet
76
from atcodertools.client.models.problem_content import ProblemContent, InputFormatDetectionError, SampleDetectionError
7+
from atcodertools.common.judgetype import ErrorType, NormalJudge, DecimalJudge, Judge
88
from atcodertools.common.logging import logger
9-
from atcodertools.common.judgetype import JudgeType, ErrorType, NormalJudge, DecimalJudge
9+
from atcodertools.constprediction.models.problem_constant_set import ProblemConstantSet
1010

1111

1212
class YesNoPredictionFailedError(Exception):
@@ -106,9 +106,10 @@ def predict_yes_no(html: str) -> Tuple[Optional[str], Optional[str]]:
106106
return yes_str, no_str
107107

108108

109-
def predict_judge_type(html: str) -> Optional[JudgeType]:
109+
def predict_judge_method(html: str) -> Optional[Judge]:
110110
def normalize(sentence):
111-
return sentence.replace('\\', '').replace("{", "").replace("}", "").replace(",", "").replace(" ", "").lower().strip()
111+
return sentence.replace('\\', '').replace("{", "").replace("}", "").replace(",", "").replace(" ", "").replace(
112+
"−", "-").lower().strip()
112113

113114
soup = BeautifulSoup(html, "html.parser")
114115
sentences = soup.get_text().split("\n")
@@ -139,16 +140,13 @@ def normalize(sentence):
139140
return None
140141

141142
if len(decimal_val_cands) == 1:
142-
if is_absolute:
143-
if is_relative:
144-
error_type = ErrorType.AbsoluteOrRelative
145-
else:
146-
error_type = ErrorType.Absolute
143+
if is_absolute and is_relative:
144+
error_type = ErrorType.AbsoluteOrRelative
145+
elif is_absolute:
146+
error_type = ErrorType.Absolute
147147
else:
148-
if is_relative:
149-
error_type = ErrorType.Relative
150-
else:
151-
assert(False)
148+
assert is_relative
149+
error_type = ErrorType.Relative
152150

153151
return DecimalJudge(error_type, 10.0**(int(list(decimal_val_cands)[0])))
154152

@@ -171,10 +169,10 @@ def predict_constants(html: str) -> ProblemConstantSet:
171169
mod = None
172170

173171
try:
174-
judge_type = predict_judge_type(html)
172+
judge = predict_judge_method(html)
175173
except MultipleModCandidatesError as e:
176174
logger.warning("decimal prediction failed -- "
177175
"two or more candidates {} are detected as decimal values".format(e.cands))
178-
judge_type = NormalJudge()
176+
judge = NormalJudge()
179177

180-
return ProblemConstantSet(mod=mod, yes_str=yes_str, no_str=no_str, judge_type=judge_type)
178+
return ProblemConstantSet(mod=mod, yes_str=yes_str, no_str=no_str, judge_method=judge)

‎atcodertools/constprediction/models/problem_constant_set.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
from atcodertools.common.judgetype import JudgeType
1+
from atcodertools.common.judgetype import Judge
22

33

44
class ProblemConstantSet:
@@ -7,9 +7,9 @@ def __init__(self,
77
mod: int = None,
88
yes_str: str = None,
99
no_str: str = None,
10-
judge_type: JudgeType = None,
10+
judge_method: Judge = None,
1111
):
1212
self.mod = mod
1313
self.yes_str = yes_str
1414
self.no_str = no_str
15-
self.judge_type = judge_type
15+
self.judge_method = judge_method

‎atcodertools/executils/run_program.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,10 +21,10 @@ def __init__(self, status: ExecStatus, output: str = None, stderr: str = None, e
2121
else:
2222
self.elapsed_ms = None
2323

24-
def is_correct_output(self, answer_text, judge_type):
24+
def is_correct_output(self, answer_text, judge_method):
2525
if self.status != ExecStatus.NORMAL:
2626
return False
27-
return judge_type.verify(self.output, answer_text)
27+
return judge_method.verify(self.output, answer_text)
2828

2929
def has_stderr(self):
3030
return len(self.stderr) > 0

‎atcodertools/tools/envgen.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -139,7 +139,7 @@ def emit_info(text):
139139
config.etc_config.in_example_format.replace("{}", "*"),
140140
config.etc_config.out_example_format.replace("{}", "*"),
141141
lang,
142-
constants.judge_type,
142+
constants.judge_method,
143143
).save_to(metadata_path)
144144
emit_info("Saved metadata to {}".format(metadata_path))
145145

‎atcodertools/tools/models/metadata.py

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,20 @@
11
import json
22

33
from atcodertools.client.models.problem import Problem
4+
from atcodertools.common.judgetype import NormalJudge, DecimalJudge, Judge
45
from atcodertools.common.language import Language
5-
from atcodertools.common.judgetype import NormalJudge, DecimalJudge
66

77

88
class Metadata:
99

10-
def __init__(self, problem: Problem, code_filename: str, sample_in_pattern: str, sample_out_pattern: str, lang: Language, judge_type=NormalJudge()):
10+
def __init__(self, problem: Problem, code_filename: str, sample_in_pattern: str, sample_out_pattern: str,
11+
lang: Language, judge_method: Judge = NormalJudge()):
1112
self.problem = problem
1213
self.code_filename = code_filename
1314
self.sample_in_pattern = sample_in_pattern
1415
self.sample_out_pattern = sample_out_pattern
1516
self.lang = lang
16-
self.judge_type = judge_type
17+
self.judge_method = judge_method
1718

1819
def to_dict(self):
1920
return {
@@ -22,29 +23,29 @@ def to_dict(self):
2223
"sample_in_pattern": self.sample_in_pattern,
2324
"sample_out_pattern": self.sample_out_pattern,
2425
"lang": self.lang.name,
25-
"judge": self.judge_type.to_dict(),
26+
"judge": self.judge_method.to_dict(),
2627
}
2728

2829
@classmethod
2930
def from_dict(cls, dic):
3031
if "judge" in dic:
3132
judge_type = dic["judge"]["judge_type"]
3233
if judge_type == "normal":
33-
judge = NormalJudge.from_dict(dic["judge"])
34+
judge_method = NormalJudge.from_dict(dic["judge"])
3435
elif judge_type == "decimal":
35-
judge = DecimalJudge.from_dict(dic["judge"])
36+
judge_method = DecimalJudge.from_dict(dic["judge"])
3637
else:
3738
raise Exception("invalid judge type")
3839
else:
39-
judge = NormalJudge()
40+
judge_method = NormalJudge()
4041

4142
return Metadata(
4243
problem=Problem.from_dict(dic["problem"]),
4344
code_filename=dic["code_filename"],
4445
sample_in_pattern=dic["sample_in_pattern"],
4546
sample_out_pattern=dic["sample_out_pattern"],
4647
lang=Language.from_name(dic["lang"]),
47-
judge_type=judge
48+
judge_method=judge_method
4849
)
4950

5051
@classmethod

0 commit comments

Comments
 (0)