Author SHA1 Message Date
jylpah d29f092bff remove unused code 2023-09-11 20:53:33 +03:00
jylpah 87d8b9a19d REFACTOR: FileQueue(), use Path for Win compat 2023-09-11 20:53:27 +03:00
jylpah de85e95f96 Fix replay search with deep dir trees 2023-09-10 23:33:55 +03:00
jylpah f1a876ea4b update 10.2 2023-09-10 23:33:24 +03:00
jylpah 5628cd4f7e WG(): Use blitzutils.Maps() 2023-09-10 23:33:12 +03:00
jylpah b001bb44be remove legacy scripts, use Jylpah/blitz-replays 2023-09-10 20:02:59 +03:00
jylpah 555aa1ae3e Update 10.2, new tankopedia format 2023-09-10 20:01:07 +03:00
jylpah 2a6ee87673 Use blitzutils.WGApiTankopedia() 2023-09-10 20:00:23 +03:00
jylpah c27d7ce4fc rename legacy blitzutils 2023-09-10 19:58:39 +03:00
jylpah 82fa381720 add: blitz-utils 2023-09-10 18:03:46 +03:00
Jylpah da8efce128 Merge pull request #36 from Jylpah/master
Mix fixes
2023-09-10 17:48:11 +03:00
jylpah b5efc4bc9d commit mix fixes 2023-09-10 17:47:13 +03:00
jylpah 859c7aa293 update tanks.json 2023-09-10 17:47:13 +03:00
jylpah b5621991c1 ignore .venv 2023-09-10 17:47:13 +03:00
jylpah b273fcd9f3 sort_dict(): force input to str 2023-09-10 17:47:13 +03:00
Jylpah d4a35e6e8d Whitespace 2023-09-10 17:47:13 +03:00
Jylpah 3f4e9c42d1 FIX Alpenstadt 2023-09-10 17:47:13 +03:00
Jylpah e4f746ca38 Update 9.7 2023-09-10 17:47:13 +03:00
Jylpah ffc006ea05 update 9.6 2023-09-10 17:47:13 +03:00
Jylpah 2b28319271 fix typo 2023-07-09 20:46:11 +03:00
Jylpah 926ef711e7 Updates 10.0 and 10.1 2023-07-09 20:45:55 +03:00
Jylpah 55a1eb65c5 Update 4.10: replays were added 2023-05-01 21:16:44 +03:00
Jylpah 4ca2ad5b69 Update 9.8 2023-05-01 21:14:29 +03:00
Jylpah becc608f0c release 9.7 2023-02-15 14:18:25 +02:00
Jylpah 97e9cf5a54 Add releases 9.4 - 9.6 2023-01-11 00:41:44 +02:00
Jylpah f5e67798b4 get_tank_type(), get_tank_id(): remove async 2022-10-30 15:46:33 +02:00
Jylpah 2eced2eda0 Update 9.3 2022-10-30 15:13:28 +02:00
Jylpah 025a786bdd Merge pull request #35 from Jylpah/dev-fix-wi-rate-limit
Dev fix replays.wotinspector.com rate limit
2022-10-29 22:22:30 +03:00
Jylpah bca474218e WoTinspector.read_replay_id(): added back 2022-10-29 22:21:14 +03:00
Jylpah e48759ce79 ThrottledClientSession(), WoTinspector(): updated 2022-10-29 22:14:33 +03:00
Jylpah 5c694fd721 Typing + remove unsed imports 2022-10-29 22:13:52 +03:00
Jylpah 1f41901042 update 9.2 2022-10-09 00:18:30 +03:00
Jylpah d275914b32 Update .gitignore 2022-08-18 09:44:39 +03:00
Jylpah 9f8c82f434 Stream client support 2022-08-18 09:42:55 +03:00
Jylpah 3a8f065f35 Update 9.2 2022-08-18 09:42:32 +03:00
Jylpah 0cac200f06 Update 9.1 2022-07-14 11:25:38 +03:00
Jylpah 342b90bcf9 SSL -> TLS 2022-06-09 20:48:50 +03:00
Jylpah a9845b6fbb Update 9.0 2022-06-08 17:25:06 +03:00
Jylpah ee36e9192f Update 8.10 2022-04-30 21:43:24 +03:00
Jylpah 5d5821564e update 8.9 2022-03-24 15:44:48 +02:00
Jylpah 3c6f9dcae3 Merge branch 'master' of https://github.com/Jylpah/blitz-tools 2022-03-24 15:41:26 +02:00
Jylpah 433e2295c2 Update 8.9 2022-03-24 15:41:22 +02:00
Jylpah 3dc9533c6a Update maps.json 2022-03-24 15:41:15 +02:00
Jylpah 83456ae150 Add updates 8.5 - 8.8 2022-02-16 17:48:01 +02:00
Jylpah 0c25c781a9 Update 8.8 2022-02-16 17:41:50 +02:00
Jylpah 244ae314d8 Update 8.8 2022-02-16 17:40:17 +02:00
Jylpah 33b0efb48d Update 8.6 2022-01-22 22:27:34 +02:00
Jylpah 574fb95f00 Update 8.6 2021-12-20 10:15:47 +02:00
Jylpah fdc5db966e Update installation and usage instructions 2021-11-24 10:28:32 +02:00
Jylpah 65e4fd83d8 Update installation instructions 2021-11-24 09:48:39 +02:00
Jylpah e9451c7f5d Merge pull request #34 from Jylpah/stable-filters-update2
Fix --filters and improve performance when filtering
2021-11-23 21:42:45 +02:00
Jylpah fa65872ad2 --filters: Bug fix and performance improvement 2021-11-23 21:42:09 +02:00
Jylpah b2f37ebcf5 Merge branch 'stable-filter-update' into stable-filters-update2 2021-11-23 18:44:09 +02:00
Jylpah d323f11662 Update analyze_wotb_replays.py 2021-11-23 18:38:49 +02:00
Jylpah d847a394a9 Add allies/enemies categorizations: xxx_damage_dealt, xxx_wins, process_battle_results(): Fix bug filter(s) 2021-11-23 18:20:11 +02:00
Jylpah 8514e9db17 Update analyze_wotb_replays.py 2021-11-23 18:08:24 +02:00
Jylpah 521986adf1 Update 8.5 2021-11-23 18:01:17 +02:00
Jylpah 7e39fdc442 Fix --filters bug with number categorization 2021-11-23 09:46:04 +02:00
Jylpah 14ee570ee5 Merge branch 'master' of https://github.com/Jylpah/blitz-tools 2021-10-29 21:18:27 +03:00
Jylpah e0228a4bf1 Fix shebang 2021-10-29 21:18:24 +03:00
Jylpah 2a93cfbce3 update 8.4 2021-10-24 21:19:06 +03:00
Jylpah 78a959224c Update 8.4 2021-10-23 22:07:23 +03:00
Jylpah ae24ab35f7 Updates 8.0 - 8.3 added 2021-09-17 10:44:38 +03:00
Jylpah ba6b4781e5 Update 8.3 2021-09-16 17:31:10 +03:00
Jylpah 050bd99488 get_filter_categories(): fix, add room_type=5: Quick Tournament(?) 2021-09-10 21:03:41 +03:00
Jylpah eb39c9ce84 Update 8.2 2021-08-22 21:21:42 +03:00
Jylpah 24547763e9 replayWorker(): Better warning text to incomplete replays 2021-08-13 13:43:17 +03:00
Jylpah 28347e1653 Fix printing of player/allies/enemies average damage columns 2021-08-07 16:42:19 +03:00
Jylpah ffe5afbf1e getTitle(): More robust tank names based on tank_ids, not UsrStrs that keep changing. 2021-08-07 16:40:42 +03:00
Jylpah 17ac09afde Update analyze_wotb_replays.py 2021-08-07 16:26:25 +03:00
Jylpah 9117963085 filter_results() bug fix 2021-08-03 19:14:44 +03:00
Jylpah dc8f9a233d Merge pull request #32 from Jylpah/master-platoon
help_extended(): change order of filter helps
2021-08-03 12:43:59 +03:00
Jylpah ce3f9993c5 help_extended(): change order of filter helps 2021-08-03 12:43:43 +03:00
Jylpah fd7eb8261d Merge pull request #31 from Jylpah/master-platoon
Added platoon rate and categorization by platoon/solo games
2021-08-03 12:43:19 +03:00
Jylpah 748ed7806b Added platoon rate and categorization by platoon/solo games 2021-08-03 12:40:36 +03:00
Jylpah 1ada279728 Update Lupus tanks.json 2021-07-27 18:47:08 +03:00
Jylpah 57cbc5b233 Tanks sorted by tank_id 2021-07-15 20:22:46 +03:00
Jylpah 43842dc140 Update 8.1 2021-07-15 20:21:53 +03:00
Jylpah 170a801652 sort_dict(): added 2021-07-15 20:21:32 +03:00
Jylpah 210e943157 Update 8.0 2021-06-08 16:57:42 +03:00
Jylpah b3b7b81eb4 BattleCategory(): add 'hits_bounced' result field 2021-06-03 15:02:27 +03:00
Jylpah 446e289041 Add --mode dmg_received, improved --mode help, new fields 2021-06-03 14:48:17 +03:00
Jylpah f8a0fdfbe1 Merge pull request #30 from Jylpah/stable-export-replays
Stable export replays
2021-05-12 00:16:42 +03:00
Jylpah 4c0de49077 name2tank_str(): error() -> debug() 2021-05-12 00:05:52 +03:00
Jylpah 07aa7af32f bug fix 2021-05-12 00:04:36 +03:00
Jylpah ea3306ee6c WoTinspector.post_replay(): add _id 2021-05-12 00:04:21 +03:00
Jylpah e4882d70af export replays(): --replays option to export replays 2021-05-11 20:55:13 +03:00
Jylpah 433c26d2f8 get_date_str(), get_replay_filename(): added 2021-05-11 20:54:39 +03:00
Jylpah f6b1b1353f Update analyze_wotb_replays.py 2021-05-11 14:14:10 +03:00
12 changed files with 4789 additions and 9546 deletions

No files matched your search

Executable
+16
View File
@@ -0,0 +1,16 @@
__pycache__/
BlitzApp
tmp/
ssl/
*.ini
*.swp
*.code-workspace
*.sqlite3
.vscode/
*.sqlite3-journal
.gitattributes
export*
*.csv
!/stats/*.csv
!*.py
/.venv
+25 -23
View File
@@ -1,5 +1,5 @@
# blitz-tools
Python scripts to analyze WoT Blitz replays and stats. All the scripts require [Python 3.7](https://www.python.org/downloads/) or higher. I am using 3.8 myself so you may have to change the Python interpreter version in the scripts' first line.
Python scripts to analyze WoT Blitz replays and stats. All the scripts require [Python 3.7](https://www.python.org/downloads/) or later.
### Replays
* [upload_wotb_replays.py](upload_wotb_replays.py): An asynchronous replay uploader.
@@ -13,46 +13,48 @@ Python scripts to analyze WoT Blitz replays and stats. All the scripts require [
* [blitzutils.py](blitzutils.py): A utils class for various Blitz related to functions. *Other scripts use this*, so you need to place it to the same folder with the other scripts
### Data
* [tanks.json](tanks.json): Tankopedia in WG JSON format extracted from the Blitz app files with [extract_tankopedia.py]](extractTankopedia.py]). Contains only limited information required by the scripts.
* [maps.json](maps.json): Blitz map names extracted from the Blitz app files with [extract_tankopedia.py]](extract_tankopedia.py]).
* [tanks.json](tanks.json): Tankopedia in WG JSON format extracted from the Blitz app files with [extract_tankopedia.py](extractTankopedia.py). Contains only limited information required by the scripts.
* [maps.json](maps.json): Blitz map names extracted from the Blitz app files with [extract_tankopedia.py](extract_tankopedia.py).
### Quick instructions
1. Install [Python 3.8](https://www.python.org/downloads/) on to your computer/laptop
2. Install the required Python modules: motor, aiohttp, aioconsole, aiofiles, aiosqlite, beautifulsoup4, lxml, xmltodict, progress
1. Install [Python 3.8](https://www.python.org/downloads/) or later to your computer/laptop
2. Install the required Python modules (see [requirements.txt](requirements.txt): motor, aiohttp, aioconsole, aiofiles, aiosqlite, beautifulsoup4, lxml, xmltodict, progress
```
# Remember to run the command with a Python interpreter >= 3.7
python -m pip install motor aiohttp aioconsole aiofiles aiosqlite beautifulsoup4 lxml xmltodict progress
python -m pip install -r requirements.txt
```
3. Download all the files including the [maps.json](maps.json) and [tanks.json](tanks.json) files. **The recommened way** is to install the scipts via `git clone`: [GitHub Desktop](https://desktop.github.com/) is an excellent GUI for git. This makes it easy to receive the updates via `git pull`.
4. Upload your replays.
3. Clone the repository. [GitHub Desktop](https://desktop.github.com/) is an excellent GUI for `git`. This makes it easy to receive the updates via `git pull`.
```
# Windows, run directly from Command Prompt or Powershell
# Check the options
git clone https://github.com/Jylpah/blitz-tools.git
```
4. Upload your replays with `upload_wotb_replays.py`
```
## Check the options
# Windows Command Prompt or Powershell
python.exe .\upload_wotb_replays.py --help
# Syntax
python.exe .\upload_wotb_replays.py REPLAY_FILE.wotbreplay
# Linux or Windows Subsystem for Linux (WSL)
./upload_wotb_replays.py --help
# Upload replays (Windows)
python.exe .\upload_wotb_replays.py REPLAY_FILE.wotbreplay DIR_WITH_REPLAYS
# example
python.exe .\upload_wotb_replays.py 20200223_0122__jylpah_T-54_grossberg.wotbreplay
# Linux
./upload_wotb_replays.py --help
```
5. Analyze replays.
5. Analyze replays with `analyze_wotb_replays.py`
```
# Windows, run directly from Command Prompt or Powershell
## Check options
# Windows Command Prompt or Powershell
python.exe .\analyze_wotb_replays.py --help
# Syntax
python.exe .\analyze_wotb_replays.py REPLAY_FILE.wotbreplay.json
# Syntax (Windows)
python.exe .\analyze_wotb_replays.py REPLAY_FILE.wotbreplay.json [DIR_WITH_REPLAYS ...]
# Example
# Example (Windows)
python.exe .\analyze_wotb_replays.py 20200223_0122__jylpah_T-54_grossberg.wotbreplay.json
# Linux
# Linux or Windows Subsystem for Linux (WSL)
./analyze_wotb_replays.py --help
```
+1668 -890
View File
File diff suppressed because it is too large. Load diff
-2065
View File
File diff suppressed because it is too large. Load diff
Executable
+2715
View File
File diff suppressed because it is too large. Load diff
-225
View File
@@ -1,225 +0,0 @@
#!/usr/bin/env python3
## PYTHON VERSION MUST BE 3.7 OR HIGHER
# Extract tankopedia data in WG API JSON format from Blitz app files (Android APK unzipped)
import sys, argparse, json, os, inspect, aiohttp, asyncio, aiofiles, re, logging, time, xmltodict, collections, configparser
import blitzutils as bu
from blitzutils import WG
logging.getLogger("asyncio").setLevel(logging.DEBUG)
FILE_CONFIG = 'blitzstats.ini'
BLITZAPP_STRINGS='/assets/Data/Strings/en.yaml'
BLITZAPP_VEHICLES_DIR='/assets/Data/XML/item_defs/vehicles/'
BLITZAPP_VEHICLE_FILE='/list.xml'
wg = None
## main() -------------------------------------------------------------
async def main(argv):
global wg
# set the directory for the script
os.chdir(os.path.dirname(sys.argv[0]))
## Read config
BLITZAPP_FOLDER = '.'
try:
if os.path.isfile(FILE_CONFIG):
config = configparser.ConfigParser()
config.read(FILE_CONFIG)
configOptions = config['EXTRACT_TANKOPEDIA']
BLITZAPP_FOLDER = configOptions.get('blitz_app_dir', BLITZAPP_FOLDER)
except:
pass
parser = argparse.ArgumentParser(description='Extract Tankopedia data from Blitz game files')
parser.add_argument('blitz_app_base', type=str, nargs='?', metavar="BLITZAPP_FOLDER", default=BLITZAPP_FOLDER, help='Base dir of the Blitz App files')
parser.add_argument('tanks', type=str, default='tanks.json', nargs='?', metavar="TANKS_FILE", help='File to write Tankopedia')
parser.add_argument('maps', type=str, default='maps.json', nargs='?', metavar='MAPS_FILE', help='File to write map names')
arggroup = parser.add_mutually_exclusive_group()
arggroup.add_argument('-d', '--debug', action='store_true', default=False, help='Debug mode')
arggroup.add_argument('-v', '--verbose', action='store_true', default=False, help='Verbose mode')
arggroup.add_argument('-s', '--silent', action='store_true', default=False, help='Silent mode')
args = parser.parse_args()
bu.set_log_level(args.silent, args.verbose, args.debug)
wg = WG()
tasks = []
for nation in wg.NATION:
tasks.append(asyncio.create_task(extract_tanks(args.blitz_app_base, nation)))
tanklist = []
for tanklist_tmp in await asyncio.gather(*tasks):
tanklist.extend(tanklist_tmp)
tank_strs, map_strs = await read_user_strs(args.blitz_app_base)
json_data = None
userStrs = {}
tanks = {}
if os.path.exists(args.tanks):
try:
async with aiofiles.open(args.tanks) as infile:
json_data = json.loads(await infile.read())
userStrs = json_data['userStr']
tanks = json_data['data']
except Exception as err:
bu.error('Unexpected error when reading file: ' + args.tanks + ' : ' + str(err))
async with aiofiles.open(args.tanks, 'w', encoding="utf8") as outfile:
new_tanks, new_userStrs = await convert_tank_names(tanklist, tank_strs)
# merge old and new tankopedia
tanks.update(new_tanks)
userStrs.update(new_userStrs)
tankopedia = collections.OrderedDict()
tankopedia['status'] = 'ok'
tankopedia['meta'] = { "count": len(tanks) }
tankopedia['data'] = tanks
tankopedia['userStr'] = userStrs
bu.verbose_std('New tankopedia \'' + args.tanks + '\' contains ' + str(len(tanks)) + ' tanks')
bu.verbose_std('New tankopedia \'' + args.tanks + '\' contains ' + str(len(userStrs)) + ' tank strings')
await outfile.write(json.dumps(tankopedia, ensure_ascii=False, indent=4, sort_keys=False))
if args.maps != None:
maps = {}
if os.path.exists(args.maps):
try:
async with aiofiles.open(args.maps) as infile:
maps = json.loads(await infile.read())
except Exception as err:
bu.error('Unexpected error when reading file: ' + args.maps + ' : ' + str(err))
# merge old and new map data
maps.update(map_strs)
async with aiofiles.open(args.maps, 'w', encoding="utf8") as outfile:
bu.verbose_std('New maps file \'' + args.maps + '\' contains ' + str(len(maps)) + ' maps')
await outfile.write(json.dumps(maps, ensure_ascii=False, indent=4, sort_keys=True))
return None
async def extract_tanks(blitz_app_base : str, nation: str):
tanks = []
list_xml_file = blitz_app_base + BLITZAPP_VEHICLES_DIR + nation + BLITZAPP_VEHICLE_FILE
if not os.path.isfile(list_xml_file):
print('ERROR: cannot open ' + list_xml_file)
return None
bu.debug('Opening file: ' + list_xml_file + ' (Nation: ' + nation + ')')
async with aiofiles.open(list_xml_file, 'r', encoding="utf8") as f:
try:
tankList = xmltodict.parse(await f.read())
for data in tankList['root'].keys():
tank_xml = tankList['root'][data]
tank = dict()
tank['is_premium'] = issubclass(type(tank_xml['price']), dict)
tank['nation'] = nation
tank['tank_id'] = await get_tank_id(nation, int(tank_xml['id']))
tank['tier'] = int(tank_xml['level'])
tank['type'] = await get_tank_type(tank_xml['tags'])
tank['userStr'] = tank_xml['userString']
#bu.debug('Reading tank string: ' + tank['userStr'], force=True)
tanks.append(tank)
except Exception as err:
bu.error(err)
sys.exit(2)
return tanks
async def read_user_strs(blitz_app_base : str) -> dict:
"""Read user strings to convert map and tank names"""
tank_strs = {}
map_strs = {}
filename = blitz_app_base + BLITZAPP_STRINGS
bu.debug('Opening file: ' + filename + ' for reading UserStrings')
try:
async with aiofiles.open(filename, 'r', encoding="utf8") as f:
p_tank = re.compile('^"(#\\w+?_vehicles:.+?)": "(.+)"$')
p_map = re.compile('^"#maps:(.+?):.+?: "(.+?)"$')
async for l in f:
m = p_tank.match(l)
if m != None:
tank_strs[m.group(1)] = m.group(2)
m = p_map.match(l)
if m != None and m.group(2) != 'Macragge':
map_strs[m.group(1)] = m.group(2)
except Exception as err:
bu.error(err)
sys.exit(1)
return tank_strs, map_strs
async def convert_tank_names(tanklist : list, tank_strs: dict) -> dict:
"""Convert tank names for Tankopedia"""
tankopedia = {}
userStrs = {}
try:
for tank in tanklist:
tank['name'] = tank_strs[tank['userStr']]
#userStrs[tank['userStr'].split(':')[1]] = tank['name']
tank.pop('userStr', None)
tank_tmp = collections.OrderedDict()
for key in sorted(tank.keys()):
tank_tmp[key] = tank[key]
tankopedia[str(tank['tank_id'])] = tank_tmp
for tank_str in tank_strs:
skip = False
key = tank_str.split(':')[1]
bu.debug('Tank string: ' + key + ' = ' + tank_strs[tank_str])
re_strs = [r'^Chassis_', r'^Turret_', r'^_', r'_short$' ]
for re_str in re_strs:
p = re.compile(re_str)
if p.match(key):
skip = True
break
if skip:
continue
userStrs[key] = tank_strs[tank_str]
# sorting
tankopedia_sorted = collections.OrderedDict()
for tank_id in sorted(tankopedia.keys(), key=int):
tankopedia_sorted[str(tank_id)] = tankopedia[str(tank_id)]
userStrs_sorted = collections.OrderedDict()
for userStr in sorted(userStrs.keys()):
userStrs_sorted[userStr] = userStrs[userStr]
# bu.debug('Tank strings: ' + str(len(userStrs_sorted)))
except Exception as err:
bu.error(err)
sys.exit(1)
return tankopedia_sorted, userStrs_sorted
async def get_tank_id(nation: str, tank_id : int) -> int:
return (tank_id << 8) + (wg.NATION_ID[nation] << 4) + 1
async def get_tank_type(tagstr : str):
tags = tagstr.split(' ')
for t_type in wg.TANK_TYPE:
if tags[0] == t_type:
return t_type
return None
### main()
if __name__ == "__main__":
# To avoid 'Event loop is closed' RuntimeError due to compatibility issue with aiohttp
if sys.platform.startswith("win") and sys.version_info >= (3, 8):
try:
from asyncio import WindowsSelectorEventLoopPolicy
except ImportError:
pass
else:
if not isinstance(asyncio.get_event_loop_policy(), WindowsSelectorEventLoopPolicy):
asyncio.set_event_loop_policy(WindowsSelectorEventLoopPolicy())
#asyncio.run(main(sys.argv[1:]), debug=True)
asyncio.run(main(sys.argv[1:]))
-91
View File
@@ -1,91 +0,0 @@
#!/usr/local/bin/python3.7
# Get Tankpedia from WoTinspector.com
import sys
import argparse
import json
import os
import inspect
import aiohttp
import asyncio
import re
import logging
from pathlib import Path
import blitzutils as bu
from blitzutils import WG
from blitzutils import WoTinspector
logging.getLogger("asyncio").setLevel(logging.DEBUG)
VERBOSE = False
DEBUG = False
tank_type = ['lightTank', 'mediumTank', 'heavyTank', 'AT-SPG']
# @asyncio.coroutine
async def get_tankopedia(filename: str):
url = "https://wotinspector.com/static/armorinspector/tank_db_blitz.js"
async with aiohttp.ClientSession() as client:
async with client.get(url) as r:
if r.status == 200:
WI_tank_db = await r.text()
WI_tank_db = WI_tank_db.split("\n")
else:
print('Error: Could not get valid HTTPS response. HTTP: ' + str(r.status) )
sys.exit(1)
tanks = {}
n = 0
p = re.compile('\\s*(\\d+):\\s{"en":"([^"]+)",.*?"tier":(\\d+), "type":(\\d), "premium":(\\d).*')
for line in WI_tank_db[1:-1]:
m = p.match(line)
tank = {}
tank['tank_id'] = int(m.group(1))
tank['name'] = m.group(2)
tank['tier'] = int(m.group(3))
tank['type'] = tank_type[int(m.group(4))]
tank['is_premium'] = (int(m.group(5)) == 1)
tanks[str(m.group(1))] = tank
n += 1
tankopedia = {}
tankopedia['status'] = "ok"
tankopedia['meta'] = {"count" : n}
tankopedia['data'] = tanks
print("Tankopedia has " + str(n) + " tanks in: " + filename)
with open(filename,'w') as outfile:
outfile.write(json.dumps(tankopedia, ensure_ascii=False, indent=4, sort_keys=True))
return None
async def main(argv):
# set the directory for the script
os.chdir(os.path.dirname(sys.argv[0]))
parser = argparse.ArgumentParser(description='Retrieve Tankopedia from WoTinspector.com')
parser.add_argument('--file', dest="outfile", help='Write Tankopedia to file')
parser.add_argument('-d', '--debug', action='store_true', default=False, help='Debug mode')
parser.add_argument('-v', '--verbose', action='store_true', default=False, help='Verbose mode')
parser.add_argument('-s', '--silent', action='store_true', default=False, help='Silent mode')
args = parser.parse_args()
bu.set_log_level(args.silent, args.verbose, args.debug)
wi = WoTinspector()
await wi.get_tankopedia(args.outfile)
await wi.close()
# main()
if __name__ == "__main__":
# To avoid 'Event loop is closed' RuntimeError due to compatibility issue with aiohttp
if sys.platform.startswith("win") and sys.version_info >= (3, 8):
try:
from asyncio import WindowsSelectorEventLoopPolicy
except ImportError:
pass
else:
if not isinstance(asyncio.get_event_loop_policy(), WindowsSelectorEventLoopPolicy):
asyncio.set_event_loop_policy(WindowsSelectorEventLoopPolicy())
asyncio.run(main(sys.argv[1:]), debug=False)
+1 -58
View File
@@ -1,58 +1 @@
{
"Random": "Random map",
"amigosville": "Falls Creek",
"amigosville_02": "Falls Creek - Bridge",
"amigosville_03": "Falls Creek - Factory",
"asia": "Lost Temple",
"canal": "Canal",
"canal_01": "Canal - Factory",
"canyon": "Canyon",
"desert_train": "Desert Sands",
"desert_train_02": "Desert Sands - Town",
"desert_train_03": "Desert Sands - Dunes",
"erlenberg": "Middleburg",
"erlenberg_01": "Middleburg - Town",
"erlenberg_02": "Middleburg - Hill",
"faust": "Faust",
"forgecity": "New Bay",
"fort": "Fort Despair",
"grossberg": "Dynasty's Pearl",
"himmelsdorf": "Himmelsdorf",
"holmeisk": "Wasteland",
"iceworld": "Everfrost",
"idle": "Yukon",
"italy": "Vineyards",
"karelia": "Rockfield",
"karieri": "Copperfield",
"lake": "Mirage",
"lumber": "Alpenstadt",
"lumber_01": "Alpenstadt - Town",
"malinovka": "Winter Malinovka",
"medvedkovo": "Dead Rail",
"medvedkovo_02": "Dead Rail - Valley",
"medvedkovo_03": "Dead Rail - Railroad",
"milbase": "Yamato Harbor",
"milbase_02": "Yamato Harbor - Battleship",
"milbase_03": "Yamato Harbor - Hills",
"milbase_04": "Yamato Harbor - Center",
"milbase_05": "Yamato Harbor - Dock",
"moon": "Sea of Tranquility",
"mountain": "Black Goldville",
"neptune": "Normandy",
"neptune_01": "Normandy - Hills",
"neptune_02": "Normandy - Beach",
"north": "North",
"ordeal": "Trial by Fire",
"pliego": "Castilla",
"port": "Port Bay",
"rift": "Hellas",
"rock": "Mayan Ruins",
"rudniki": "Mines",
"rudniki_01": "Mines - Hill",
"rudniki_02": "Mines - Village",
"rudniki_03": "Mines - Center",
"savanna": "Oasis Palms",
"skit": "Naval Frontier",
"test": "World of Ducks",
"tutorial": "Proving Grounds"
}
{"Random": {"key": "Random", "name": "Random map", "id": -1, "mode": 1}, "amigosville": {"key": "amigosville", "name": "Falls Creek", "id": -1, "mode": 1}, "amigosville_02": {"key": "amigosville_02", "name": "Falls Creek - Bridge", "id": -1, "mode": 3}, "amigosville_03": {"key": "amigosville_03", "name": "Falls Creek - Factory", "id": -1, "mode": 3}, "asia": {"key": "asia", "name": "Lost Temple", "id": -1, "mode": 1}, "canal": {"key": "canal", "name": "Canal", "id": -1, "mode": 1}, "canal_01": {"key": "canal_01", "name": "Canal - Factory", "id": -1, "mode": 3}, "canyon": {"key": "canyon", "name": "Canyon", "id": -1, "mode": 1}, "desert_train": {"key": "desert_train", "name": "Desert Sands", "id": -1, "mode": 1}, "desert_train_02": {"key": "desert_train_02", "name": "Desert Sands - Town", "id": -1, "mode": 3}, "desert_train_03": {"key": "desert_train_03", "name": "Desert Sands - Dunes", "id": -1, "mode": 3}, "erlenberg": {"key": "erlenberg", "name": "Middleburg", "id": -1, "mode": 1}, "erlenberg_01": {"key": "erlenberg_01", "name": "Middleburg - Town", "id": -1, "mode": 3}, "erlenberg_02": {"key": "erlenberg_02", "name": "Middleburg - Hill", "id": -1, "mode": 3}, "faust": {"key": "faust", "name": "Faust", "id": -1, "mode": 1}, "forgecity": {"key": "forgecity", "name": "New Bay", "id": -1, "mode": 1}, "fort": {"key": "fort", "name": "Fort Despair", "id": -1, "mode": 1}, "grossberg": {"key": "grossberg", "name": "Dynasty's Pearl", "id": -1, "mode": 1}, "himmelsdorf": {"key": "himmelsdorf", "name": "Himmelsdorf", "id": -1, "mode": 1}, "holland": {"key": "holland", "name": "Molendijk", "id": -1, "mode": 1}, "holmeisk": {"key": "holmeisk", "name": "Wasteland", "id": -1, "mode": 1}, "iceworld": {"key": "iceworld", "name": "Everfrost", "id": -1, "mode": 2}, "idle": {"key": "idle", "name": "Yukon", "id": -1, "mode": 1}, "italy": {"key": "italy", "name": "Vineyards", "id": -1, "mode": 1}, "karelia": {"key": "karelia", "name": "Rockfield", "id": -1, "mode": 1}, "karieri": {"key": "karieri", "name": "Copperfield", "id": -1, "mode": 1}, "lake": {"key": "lake", "name": "Mirage", "id": -1, "mode": 1}, "lumber": {"key": "lumber", "name": "Horrorstadt", "id": -1, "mode": 1}, "lumber_01": {"key": "lumber_01", "name": "Alpenstadt - Town", "id": -1, "mode": 3}, "malinovka": {"key": "malinovka", "name": "Winter Malinovka", "id": -1, "mode": 1}, "medvedkovo": {"key": "medvedkovo", "name": "Dead Rail", "id": -1, "mode": 1}, "medvedkovo_02": {"key": "medvedkovo_02", "name": "Dead Rail - Valley", "id": -1, "mode": 3}, "medvedkovo_03": {"key": "medvedkovo_03", "name": "Dead Rail - Railroad", "id": -1, "mode": 3}, "milbase": {"key": "milbase", "name": "Yamato Harbor", "id": -1, "mode": 1}, "milbase_02": {"key": "milbase_02", "name": "Yamato Harbor - Battleship", "id": -1, "mode": 3}, "milbase_03": {"key": "milbase_03", "name": "Yamato Harbor - Hills", "id": -1, "mode": 3}, "milbase_04": {"key": "milbase_04", "name": "Yamato Harbor - Center", "id": -1, "mode": 3}, "milbase_05": {"key": "milbase_05", "name": "Yamato Harbor - Dock", "id": -1, "mode": 3}, "moon": {"key": "moon", "name": "Sea of Tranquility", "id": -1, "mode": 2}, "mountain": {"key": "mountain", "name": "Black Goldville", "id": -1, "mode": 1}, "neptune": {"key": "neptune", "name": "Normandy", "id": -1, "mode": 1}, "neptune_01": {"key": "neptune_01", "name": "Normandy - Hills", "id": -1, "mode": 3}, "neptune_02": {"key": "neptune_02", "name": "Normandy - Beach", "id": -1, "mode": 3}, "north": {"key": "north", "name": "North", "id": -1, "mode": 1}, "ordeal": {"key": "ordeal", "name": "Trial by Fire", "id": -1, "mode": 1}, "plant": {"key": "plant", "name": "Ghost Factory", "id": -1, "mode": 1}, "pliego": {"key": "pliego", "name": "Castilla", "id": -1, "mode": 1}, "port": {"key": "port", "name": "Port Bay", "id": -1, "mode": 1}, "rift": {"key": "rift", "name": "Hellas", "id": -1, "mode": 1}, "rock": {"key": "rock", "name": "Mayan Ruins", "id": -1, "mode": 1}, "rudniki": {"key": "rudniki", "name": "Mines", "id": -1, "mode": 1}, "rudniki_01": {"key": "rudniki_01", "name": "Mines - Hill", "id": -1, "mode": 3}, "rudniki_02": {"key": "rudniki_02", "name": "Mines - Village", "id": -1, "mode": 3}, "rudniki_03": {"key": "rudniki_03", "name": "Mines - Center", "id": -1, "mode": 3}, "savanna": {"key": "savanna", "name": "Oasis Palms", "id": -1, "mode": 1}, "skit": {"key": "skit", "name": "Naval Frontier", "id": -1, "mode": 1}, "test": {"key": "test", "name": "World of Ducks", "id": -1, "mode": 2}, "tutorial": {"key": "tutorial", "name": "Proving Grounds", "id": -1, "mode": 0}}
+103 -81
View File
@@ -1,83 +1,105 @@
# WoT Blitz releases
| Release | Date | Notes | Inaccurate? |
|:-------:|:--------:|-------------|-----|
| 7.9 | 2021-04-27 | Mines rework, new tutorial, colorblind mode, JgPz E100 legendary camo | |
| 7.8 | 2021-03-24 | Mid-tier balance changes, Grille line new consumables | |
| 7.7 | 2021-02-24 | Czechoslovakian medium tanks | |
| 7.6 | 2021-01-27 | AMC 13 90, TOG II buffs, free replay camera | |
| 7.5 | 2020-12-09 | ATGM removal, Batchat, Kpz70, T-2020 buff | |
| 7.4 | 2020-11-12 | TD buffs, Clan missions | |
| 7.3 | 2020-10-08 | Swedish HT line | |
| 7.2 | 2020-08-19 | Balance changes, added tier X ammo capa, new aiming reticles | |
| 7.1 | 2020-07-15 | New Yukon map, IS-6, KV-4, Patton 48 armor buff, Grille DPM buff, SP IC, AMZ 13 75, M41 Bulldog mobility buff | |
| 7.0 | 2020-06-17 | British LTs, graphics improvements, ATGM nerf, Sheridan nerf, map fixes | |
| 6.10 | 2020-04-29 | T92E1 nerf, Tier V-IX heavy HP buff, Battle Pass, map fixes | |
| 6.9 | 2020-03-25 | Japanese TDs, Tier X heavy HP buff | |
| 6.8 | 2020-02-19 | ATGM nerf, Hellas map | |
| 6.7 | 2020-01-15 | ATGM US Lights | |
| 6.6 | 2019-12-18 | T-62A + Leo buffs | |
| 6.5 | 2019-11-21 | New UI | |
| 6.4 | 2019-10-16 | Halloween map, mid-tier HT buff... | |
| 6.3 | 2019-09-18 | European line | |
| 6.2 | 2019-08-14 | New MM | |
| 6.1 | 2019-07-10 | Autoloader buff | |
| 6.0 | 2019-06-05 | UK TD line #2 | |
| 5.10 | 2019-04-24 | new missions | |
| 5.9 | 2019-03-27 | New shell tracers, minor fixes | |
| 5.8 | 2019-02-21 | Chinese TD line | |
| 5.7 | 2019-01-16 | Avatars | |
| 5.6 | 2018-12-05 | Normandy map | |
| 5.5 | 2018-11-13 | Low-tier clean-up | |
| 5.4 | 2018-10-10 | MM "rigging" adjusted | |
| 5.3 | 2018-09-12 | MM "rigging" started, Newbie queue added | |
| 5.2 | 2018-08-22 | Faust map | |
| 5.1 | 2018-07-31 | New MM: vehicle class matching | |
| 5.0 | 2018-06-30 | Vineyeards map update, Spare Parts killed | |
| 4.10 | 2018-05-24 | Alpenstadt map | |
| 4.9 | 2018-04-25 | Chinese heavy tanks | |
| 4.8 | 2018-03-28 | Choice of battle modes, crew skills accelerator | |
| 4.7 | 2018-02-28 | New minimap | |
| 4.6 | 2018-01-31 | Chinese nation | |
| 4.5 | 2017-12-19 | Dynasty's Pearl map | |
| 4.4 | 2017-11-09 | New British tanks | |
| 4.3 | 2017-10-10 | Level 7 screw skills | |
| 4.2 | 2017-09-12 | French LTs, Naval Frontier map | |
| 4.1 | 2017-08-16 | New markers | |
| 4.0 | 2017-07-13 | Rating battles | Y |
| 3.10 | 2017-06-15 | Fixes | Y |
| 3.9 | 2017-05-19 | Military Honor, Mayan Ruins | Y |
| 3.8 | 2017-04-24 | Spare Parts, New 9-slot Equipment system | Y |
| 3.7 | 2017-03-22 | Balance changes, Tournaments | Y |
| 3.6 | 2017-02-16 | Vehicle Enrichment | Y |
| 3.5 | 2017-01-13 | Boosters, Dead rail update | Y |
| 3.4 | 2016-11-28 | Canyon map, USA T57 line | |
| 3.3 | 2016-11-01 | Updated Mines map | |
| 3.2 | 2016-10-04 | Copperfield update | |
| 3.1 | 2016-09-03 | Clan tournaments, Yamato Harbor map | Y |
| 3.0 | 2016-07-20 | Supremacy mode, Falls Creek update | Y |
| 2.11 | 2016-06-28 | Vineyeards map, Oasis Palms update, German paper TDs | |
| 2.10 | 2016-06-07 | Rockfield map update, Tank birthdays, high-pen gun for IS-6 | |
| 2.9 | 2016-05-03 | Training rooms, +/-1 Matchmaker | Y |
| 2.8 | 2016-04-04 | New Russian TD line, new consumables system | Y |
| 2.7 | 2016-03-07 | Mirage map | Y |
| 2.6 | 2016-02-10 | German LTs, shooting through obstacles | Y |
| 2.5 | 2016-01-11 | Camouflages, Clan supply | Y |
| 2.4 | 2015-12-09 | Additional Japanese tanks, windows version | Y |
| 2.3 | 2015-11-12 | Canal map | Y |
| 2.2 | 2015-10-05 | Desert Sands map update | Y |
| 2.1 | 2015-09-07 | UK TDs, Clan Chat | Y |
| 2.0 | 2015-08-05 | Clans, Oasis Palms updated | |
| 1.11 | 2015-07-13 | KV-2, KV-13, Sherman Jumbo | Y |
| 1.10 | 2015-06-11 | Port Bay map | Y |
| 1.9 | 2015-05-13 | New German medium tanks (Leo) | Y |
| 1.8 | 2015-04-14 | Game tutorial, Bot map for new players | Y |
| 1.7 | 2015-02-16 | US Turreted TDs | Y |
| 1.6 | 2015-01-19 | Castilla map, UK HT line | Y |
| 1.5 | 2014-12-01 | UK Nation | Y |
| 1.4 | 2014-10-29 | German HT line (Maus) | Y |
| 1.3 | 2014-10-02 | USSR IS-4 line, Lost Temple, Black Goldville maps | Y |
| 1.2 | 2014-08-20 | Oasis Palms map | Y |
| 1.1.1 | 2014-07-23 | Fixes :-) | |
| 1.0 | 2014-06-26 | Worldwide launch | |
| 0.9 | 2014-05-05 | Nordic soft launch | |
| Release | Date | Content updates |
|:-------:|:--------:|-------------|
| 10.1 | 2023-07-05 | PBR Camos and ? |
| 10.0 | 2023-05-24 | Lagoon map, Czechoslovakian heavies, tanks start shells loaded |
| 9.8 | 2023-03-22 | F\*cked up The Quick Commands Wheel, camos |
| 9.7 | 2023-02-15 | Chinese light tanks |
| 9.6 | 2023-01-11 | Finally fixing game sounds for players with Genelec studio monitors, more gold camos |
| 9.5 | 2022-12-14 | Camos and ?|
| 9.4 | 2022-11-09 | Italian TD line |
| 9.3 | 2022-10-04 | Nothing, except legandary camos and utterly incomprehensible shell icons |
| 9.2 | 2022-08-17 | New shot sounds and shell hit marks, vehicle restoration from the game client |
| 9.1 | 2022-07-14 | Releases updated via DLC, graphics updates for Object 140, Patton 46 |
| 9.0 | 2022-06-08 | Japanies heavies, Castilla graphics update, new garage |
| 8.10 | 2022-04-28 | New battle results screen, new "Operation" mechanics |
| 8.9 | 2022-03-23 | "Free" camo resupply, VK 72.01 supplies |
| 8.8 | 2022-02-16 | Molendijk map |
| 8.7 | 2022-01-19 | Soviet Light Tanks |
| 8.6 | 2021-12-15 | Joining Light and Medium Tank classes in MM |
| 8.5 | 2021-11-17 | New national crew voices, economy update |
| 8.4 | 2021-10-20 | Polish heavy tanks |
| 8.3 | 2021-09-16 | FV4005, FV4022, VK 91.01 (P) and WZ-121 legendary camos, -5 deg gun depression to WZ-120 |
| 8.2 | 2021-08-18 | Ghost Factory map, XP/credits nerf for inactive players, new store UI |
| 8.1 | 2021-07-15 | Armor highlighting updated |
| 8.0 | 2021-06-08 | American experimental heavies (Yoh-line), PBR camos |
| 7.9 | 2021-04-27 | Mines rework, new tutorial, colorblind mode, JgPz E100 legendary camo |
| 7.8 | 2021-03-24 | Mid-tier balance changes, Grille line new consumables |
| 7.7 | 2021-02-24 | Czechoslovakian medium tanks |
| 7.6 | 2021-01-27 | AMC 13 90, TOG II buffs, free replay camera |
| 7.5 | 2020-12-09 | ATGM removal, Batchat, Kpz70, T-2020 buff |
| 7.4 | 2020-11-12 | TD buffs, Clan missions |
| 7.3 | 2020-10-08 | Swedish HT line |
| 7.2 | 2020-08-19 | Balance changes, added tier X ammo capa, new aiming reticles |
| 7.1 | 2020-07-15 | New Yukon map, IS-6, KV-4, Patton 48 armor buff, Grille DPM buff, SP IC, AMZ 13 75, M41 Bulldog mobility buff |
| 7.0 | 2020-06-17 | British LTs, graphics improvements, ATGM nerf, Sheridan nerf, map fixes |
| 6.10 | 2020-04-29 | T92E1 nerf, Tier V-IX heavy HP buff, Battle Pass, map fixes |
| 6.9 | 2020-03-25 | Japanese TDs, Tier X heavy HP buff |
| 6.8 | 2020-02-19 | ATGM nerf, Hellas map |
| 6.7 | 2020-01-15 | ATGM US Lights |
| 6.6 | 2019-12-18 | T-62A + Leo buffs |
| 6.5 | 2019-11-21 | New UI |
| 6.4 | 2019-10-16 | Halloween map, mid-tier HT buff... |
| 6.3 | 2019-09-18 | European line |
| 6.2 | 2019-08-14 | New MM |
| 6.1 | 2019-07-10 | Autoloader buff |
| 6.0 | 2019-06-05 | UK TD line #2 |
| 5.10 | 2019-04-24 | new missions |
| 5.9 | 2019-03-27 | New shell tracers, minor fixes |
| 5.8 | 2019-02-21 | Chinese TD line |
| 5.7 | 2019-01-16 | Avatars |
| 5.6 | 2018-12-05 | Normandy map |
| 5.5 | 2018-11-13 | Low-tier clean-up :-( |
| 5.4 | 2018-10-10 | MM "rigging" adjusted |
| 5.3 | 2018-09-12 | MM "rigging" started, Newbie queue added |
| 5.2 | 2018-08-22 | Faust map |
| 5.1 | 2018-07-31 | New MM: vehicle class matching |
| 5.0 | 2018-06-30 | Vineyeards map update, Spare Parts removed ! |
| 4.10 | 2018-05-24 | Replays, Alpenstadt map |
| 4.9 | 2018-04-25 | Chinese heavy tanks |
| 4.8 | 2018-03-28 | Choice of battle modes, crew skills accelerator |
| 4.7 | 2018-02-28 | New minimap |
| 4.6 | 2018-01-31 | Chinese nation |
| 4.5 | 2017-12-19 | Dynasty's Pearl map |
| 4.4 | 2017-11-09 | New British tanks |
| 4.3 | 2017-10-10 | Level 7 screw skills |
| 4.2 | 2017-09-12 | French LTs, Naval Frontier map |
| 4.1 | 2017-08-16 | New markers |
| 4.0 | 2017-07-13 | Rating battles (date uncertain) |
| 3.10 | 2017-06-15 | Fixes (date uncertain) |
| 3.9 | 2017-05-19 | Military Honor, Mayan Ruins (date uncertain) |
| 3.8 | 2017-04-24 | Spare Parts, New 9-slot Equipment system (date uncertain) |
| 3.7 | 2017-03-22 | Balance changes, Tournaments (date uncertain) |
| 3.6 | 2017-02-16 | Vehicle Enrichment (date uncertain) |
| 3.5 | 2017-01-13 | Boosters, Dead rail update (date uncertain) |
| 3.4 | 2016-11-28 | Canyon map, USA T57 line |
| 3.3 | 2016-11-01 | Updated Mines map |
| 3.2 | 2016-10-04 | Copperfield update |
| 3.1 | 2016-09-03 | Clan tournaments, Yamato Harbor map (date uncertain) |
| 3.0 | 2016-07-20 | Supremacy mode, Falls Creek update (date uncertain) |
| 2.11 | 2016-06-28 | Vineyeards map, Oasis Palms update, German paper TDs |
| 2.10 | 2016-06-07 | Rockfield map update, Tank birthdays, high-pen gun for IS-6 |
| 2.9 | 2016-05-03 | Training rooms, +/-1 Matchmaker (date uncertain) |
| 2.8 | 2016-04-04 | New Russian TD line, new consumables system (date uncertain) |
| 2.7 | 2016-03-07 | Mirage map (date uncertain) |
| 2.6 | 2016-02-10 | German LTs, shooting through obstacles (date uncertain) |
| 2.5 | 2016-01-11 | Camouflages, Clan supply (date uncertain) |
| 2.4 | 2015-12-09 | Additional Japanese tanks, windows version (date uncertain) |
| 2.3 | 2015-11-12 | Canal map (date uncertain) |
| 2.2 | 2015-10-05 | Desert Sands map update (date uncertain) |
| 2.1 | 2015-09-07 | UK TDs, Clan Chat (date uncertain) |
| 2.0 | 2015-08-05 | Clans, Oasis Palms updated |
| 1.11 | 2015-07-13 | KV-2, KV-13, Sherman Jumbo (date uncertain) |
| 1.10 | 2015-06-11 | Port Bay map (date uncertain) |
| 1.9 | 2015-05-13 | New German medium tanks (Leo) (date uncertain) |
| 1.8 | 2015-04-14 | Game tutorial, Bot map for new players (date uncertain) |
| 1.7 | 2015-02-16 | US Turreted TDs (date uncertain) |
| 1.6 | 2015-01-19 | Castilla map, UK HT line (date uncertain) |
| 1.5 | 2014-12-01 | UK Nation (date uncertain) |
| 1.4 | 2014-10-29 | German HT line (Maus) (date uncertain) |
| 1.3 | 2014-10-02 | USSR IS-4 line, Lost Temple, Black Goldville maps (date uncertain) |
| 1.2 | 2014-08-20 | Oasis Palms map (date uncertain) |
| 1.1.1 | 2014-07-23 | Fixes :-) |
| 1.0 | 2014-06-26 | Worldwide launch |
| 0.9 | 2014-05-05 | Nordic soft launch |
+1
View File
@@ -8,3 +8,4 @@ lxml
xmltodict
progress
pytest-asyncio
blitz-utils @ git+https://github.com/Jylpah/blitz-utils
+1 -5956
View File
File diff suppressed because it is too large. Load diff
+259 -157
View File
@@ -1,15 +1,15 @@
#!/usr/bin/env python3
## PYTHON VERSION MUST BE 3.7 OR HIGHER
import json, argparse, inspect, sys, os, io, base64, aiohttp, urllib, asyncio, aiofiles, aioconsole
import logging, re, concurrent.futures, configparser, motor.motor_asyncio, ssl, zipfile
import blitzutils as bu
from blitzutils import WG
from blitzutils import WoTinspector
import json, argparse, sys, os, io, asyncio, aiofiles, aioconsole
import logging, re, configparser, zipfile
import blitzutils1 as bu
from blitzutils1 import WG
from blitzutils1 import WoTinspector
logging.getLogger("asyncio").setLevel(logging.DEBUG)
FILE_CONFIG = 'blitzstats.ini'
FILE_CONFIG = "blitzstats.ini"
# DB_C_REPLAYS = 'Replays'
DEBUG = False
@@ -19,19 +19,20 @@ MAX_RETRIES = 3
REPLAY_N = 0
SKIPPED_N = 0
ERROR_N = 0
WIurl='https://wotinspector.com/api/replay/upload?'
WG_appID = '81381d3f45fa4aa75b78a7198eb216ad'
wg = None
wi = None
WIurl = "https://wotinspector.com/api/replay/upload?"
WG_appID = "81381d3f45fa4aa75b78a7198eb216ad"
wg: WG | None = None
wi: WoTinspector | None = None
async def main(argv):
async def main(argv) -> int:
global wg, wi
current_dir = os.getcwd()
# set the directory for the script
os.chdir(os.path.dirname(sys.argv[0]))
# options defaults
OPT_WORKERS_N = 5
OPT_WORKERS_N = 2
WG_ID = None
WG_ACCOUNT = None # format: nick@server, where server is either 'eu', 'ru', 'na', 'asia' or 'china'.
# China is not supported since WG API stats are not available there
@@ -40,60 +41,101 @@ async def main(argv):
try:
## Read config
if os.path.isfile(FILE_CONFIG):
bu.debug('Reading config file: ' + FILE_CONFIG)
bu.debug("Reading config file: " + FILE_CONFIG)
config = configparser.ConfigParser()
config.read(FILE_CONFIG)
try:
if 'UPLOADER' in config.sections():
configUploader = config['UPLOADER']
OPT_WORKERS_N = configUploader.getint('opt_workers', OPT_WORKERS_N)
except (KeyError, configparser.NoSectionError) as err:
bu.error(exception=err)
try:
if 'WG' in config.sections():
configWG = config['WG']
if "WG" in config.sections():
configWG = config["WG"]
# WG account id of the uploader:
# # Find it here: https://developers.wargaming.net/reference/all/wotb/account/list/
WG_ID = configWG.getint('wg_id', WG_ID)
WG_ACCOUNT = configWG.get('wg_account', WG_ACCOUNT)
WG_ID = configWG.getint("wg_id", WG_ID)
WG_ACCOUNT = configWG.get("wg_account", WG_ACCOUNT)
## WG API Rules limit 10 request / sec. Higher rate of requests will return errors ==> extra delay
WG_RATE_LIMIT = configWG.getint('wg_rate_limit', WG_RATE_LIMIT)
WG_RATE_LIMIT = configWG.getint("wg_rate_limit", WG_RATE_LIMIT)
except (KeyError, configparser.NoSectionError) as err:
bu.error(exception=err)
except Exception as err:
bu.error(exception=err)
parser = argparse.ArgumentParser(description='Post replays(s) to WoTinspector.com and retrieve replay data as JSON')
parser.add_argument('-id', dest='accountID', type=int, default=WG_ID, help='WG account_id')
parser.add_argument('-a', '--account', dest='account', type=str, default=WG_ACCOUNT, help='Uploader\'s WG account name. Format: ACCOUNT_NAME@SERVER')
parser.add_argument('-t','--title', type=str, default=None, help='Title for replays. Use "NN" for continous numbering. Default is automatic naming')
parser.add_argument('-p', '--private', dest="private", action='store_true', default=False, help='Set replays private on WoTinspector.com')
parser.add_argument('--tankopedia', type=str, default='tanks.json', help='JSON file to read Tankopedia from. Default: "tanks.json"')
parser.add_argument('--mapfile', type=str, default='maps.json', help='JSON file to read Blitz map names from. Default: "maps.json"')
parser.add_argument('-d', '--debug', action='store_true', default=False, help='Debug mode')
parser.add_argument('-v', '--verbose', action='store_true', default=False, help='Verbose mode')
parser.add_argument('-s', '--silent', action='store_true', default=False, help='Silent mode')
parser.add_argument('files', metavar='FILE1 [FILE2 ...]', type=str, nargs='+', help='Files to read. Use \'-\' for STDIN"')
parser = argparse.ArgumentParser(
description="Post replays(s) to WoTinspector.com and retrieve replay data as JSON"
)
parser.add_argument(
"-id", dest="accountID", type=int, default=WG_ID, help="WG account_id"
)
parser.add_argument(
"-a",
"--account",
dest="account",
type=str,
default=WG_ACCOUNT,
help="Uploader's WG account name. Format: ACCOUNT_NAME@SERVER",
)
parser.add_argument(
"-t",
"--title",
type=str,
default=None,
help='Title for replays. Use "NN" for continous numbering. Default is automatic naming',
)
parser.add_argument(
"-p",
"--private",
dest="private",
action="store_true",
default=False,
help="Set replays private on WoTinspector.com",
)
parser.add_argument(
"--tankopedia",
type=str,
default="tanks.json",
help='JSON file to read Tankopedia from. Default: "tanks.json"',
)
parser.add_argument(
"--mapfile",
type=str,
default="maps.json",
help='JSON file to read Blitz map names from. Default: "maps.json"',
)
parser.add_argument(
"-d", "--debug", action="store_true", default=False, help="Debug mode"
)
parser.add_argument(
"-v", "--verbose", action="store_true", default=False, help="Verbose mode"
)
parser.add_argument(
"-s", "--silent", action="store_true", default=False, help="Silent mode"
)
parser.add_argument(
"files",
metavar="FILE1 [FILE2 ...]",
type=str,
nargs="+",
help="Files to read. Use '-' for STDIN\"",
)
args = parser.parse_args()
bu.set_verbose(args.verbose)
bu.set_log_level(args.silent, args.verbose, args.debug)
wg = WG(WG_appID, args.tankopedia, args.mapfile)
wi = WoTinspector(rate_limit=WG_RATE_LIMIT)
wi = WoTinspector()
if args.account != None:
if args.account is not None:
args.accountID = await wg.get_account_id(args.account)
bu.debug('WG account_id: ' + str(args.accountID))
bu.debug("WG account_id: " + str(args.accountID))
if args.accountID == None:
if args.accountID is None:
args.accountID = 0
try:
queue = asyncio.Queue()
queue: asyncio.Queue = asyncio.Queue()
# rebase file arguments due to moving the working directory to the script location
bu.verbose_std("replays.wotinspector.com rate limits replays to 6 / minute")
args.files = bu.rebase_file_args(current_dir, args.files)
tasks = []
@@ -101,80 +143,100 @@ async def main(argv):
tasks.append(asyncio.create_task(mkReplayQ(queue, args.files, args.title)))
# Start tasks to process the Queue
for i in range(OPT_WORKERS_N):
tasks.append(asyncio.create_task(replayWorker(queue, i, args.accountID, args.private)))
bu.debug('Task ' + str(i) + ' started')
tasks.append(
asyncio.create_task(
replayWorker(queue, i, args.accountID, args.private)
)
)
bu.debug("Task " + str(i) + " started")
bu.debug('Waiting for the replay scanner to finish')
bu.debug("Waiting for the replay scanner to finish")
await asyncio.wait([tasks[0]])
bu.debug('Scanner finished. Waiting for workers to finish queue')
bu.debug("Scanner finished. Waiting for workers to finish queue")
await queue.join()
bu.debug('Cancelling workers')
bu.debug("Cancelling workers")
for task in tasks:
task.cancel()
bu.debug('Waiting for workers to cancel')
bu.debug("Waiting for workers to cancel")
await asyncio.gather(*tasks, return_exceptions=True)
bu.verbose_std(str(REPLAY_N) + ' replays: ' + str(REPLAY_N - SKIPPED_N - ERROR_N) + ' uploaded, ' + str(SKIPPED_N) + ' skipped, ' + str(ERROR_N) + ' errors')
bu.verbose_std(
str(REPLAY_N)
+ " replays: "
+ str(REPLAY_N - SKIPPED_N - ERROR_N)
+ " uploaded, "
+ str(SKIPPED_N)
+ " skipped, "
+ str(ERROR_N)
+ " errors"
)
except KeyboardInterrupt:
print('Ctrl-c pressed ...')
print("Ctrl-c pressed ...")
sys.exit(1)
finally:
## Need to close the aiohttp.session since Python destructors do not support async methods...
await wg.session.close()
await wi.close()
return None
return 0
async def mkReplayQ(queue : asyncio.Queue, files : list, title : str):
async def mkReplayQ(queue: asyncio.Queue, files: list, title: str):
"""Create queue of replays to post"""
p_replayfile = re.compile('.*\\.wotbreplay$')
if files[0] == '-':
bu.debug('reading replay file list from STDIN')
p_replayfile = re.compile(".*\\.wotbreplay$")
if files[0] == "-":
bu.debug("reading replay file list from STDIN")
stdin, _ = await aioconsole.get_standard_streams()
while True:
line = (await stdin.readline()).decode('utf-8').rstrip()
line = (await stdin.readline()).decode("utf-8").rstrip()
if not line:
break
else:
if (p_replayfile.match(line) != None):
if p_replayfile.match(line) is not None:
await queue.put(await mkQueueItem(line, title))
else:
for fn in files:
if fn.endswith('"'):
fn = fn[:-1]
if os.path.isfile(fn) and (p_replayfile.match(fn) != None):
if os.path.isfile(fn) and (p_replayfile.match(fn) is not None):
await queue.put(await mkQueueItem(fn, title))
bu.debug('File added to queue: ' + fn)
bu.debug("File added to queue: " + fn)
elif os.path.isdir(fn):
with os.scandir(fn) as dirEntry:
for entry in dirEntry:
try:
bu.debug('Found: ' + entry.name)
if entry.is_file() and (p_replayfile.match(entry.name) != None):
bu.debug("Found: " + entry.name)
if entry.is_file() and (
p_replayfile.match(entry.name) is not None
):
bu.debug(entry.name)
await queue.put(await mkQueueItem(entry.path, title))
bu.debug('File added to queue: ' + entry.path)
bu.debug("File added to queue: " + entry.path)
except Exception as err:
bu.error(str(err))
else:
bu.error('File not found: ' + fn)
bu.error("File not found: " + fn)
bu.debug('Finished')
bu.debug("Finished")
return None
async def mkQueueItem(filename : str, title : str) -> list:
async def mkQueueItem(filename: str, title: str) -> list:
"""Make an item to replay queue"""
global REPLAY_N
REPLAY_N +=1
REPLAY_N += 1
return [filename, REPLAY_N, getTitle(filename, title, REPLAY_N)]
async def replayWorker(queue: asyncio.Queue, workerID: int, account_id: int, priv = False):
async def replayWorker(
queue: asyncio.Queue, workerID: int, account_id: int, priv=False
):
"""Async Worker to process the replay queue"""
global SKIPPED_N
global ERROR_N
global wi
assert wi is not None, "wi cannot be None"
while True:
item = await queue.get()
@@ -182,20 +244,27 @@ async def replayWorker(queue: asyncio.Queue, workerID: int, account_id: int, pri
N = item[1]
title = item[2]
replay_json_fn = filename + '.json'
msg_str = 'Replay[' + str(N) + ']: '
replay_json_fn = filename + ".json"
msg_str = "Replay[" + str(N) + "]: "
#bu.debug(msg_str + replay_json_fn)
# bu.debug(msg_str + replay_json_fn)
try:
if os.path.isfile(replay_json_fn):
async with aiofiles.open(replay_json_fn) as fp:
replay_json = json.loads(await fp.read())
#if replay_json['status'] == 'ok':
# if replay_json['status'] == 'ok':
if wi.chk_JSON_replay(replay_json):
bu.verbose_std(msg_str + title + ' has already been posted. Skipping.' )
bu.verbose_std(
msg_str + title + " has already been posted. Skipping."
)
else:
os.remove(replay_json_fn)
bu.debug(msg_str + "Replay JSON not valid: Deleting " + replay_json_fn, id=workerID)
bu.debug(
msg_str
+ "Replay JSON not valid/complete: Deleting "
+ replay_json_fn,
id=workerID,
)
SKIPPED_N += 1
queue.task_done()
continue
@@ -203,131 +272,162 @@ async def replayWorker(queue: asyncio.Queue, workerID: int, account_id: int, pri
except asyncio.CancelledError as err:
raise err
except Exception as err:
bu.error(msg_str + 'Unexpected error: ' + str(type(err)) + ' : '+ str(err))
bu.error(msg_str + "Unexpected error: " + str(type(err)) + " : " + str(err))
try:
#bu.debug('Opening file [' + str(N) +']: ' + filename)
async with aiofiles.open(filename,'rb') as fp:
# bu.debug('Opening file [' + str(N) +']: ' + filename)
async with aiofiles.open(filename, "rb") as fp:
filename = os.path.basename(filename)
bu.debug(msg_str + 'File: ' + filename)
json_resp = await wi.post_replay(await fp.read(), filename, account_id, title, priv, N)
if json_resp != None:
if (await bu.save_JSON(replay_json_fn,json_resp)):
bu.debug(msg_str + "File: " + filename)
json_resp = await wi.post_replay(
await fp.read(), filename, account_id, title, priv, N
)
if json_resp is not None:
if await bu.save_JSON(replay_json_fn, json_resp):
if wi.chk_JSON_replay(json_resp):
if not bu.debug(msg_str + 'Replay saved OK: ' + filename):
bu.verbose_std(msg_str + title + ' posted')
if not bu.debug(msg_str + "Replay saved OK: " + filename):
bu.verbose_std(msg_str + title + " posted")
else:
bu.warning(msg_str +'Replay file is not valid: ' + filename)
bu.warning(
msg_str
+ "Replay file is not valid/complete: "
+ filename
)
ERROR_N += 1
else:
bu.error(msg_str + 'Error saving replay: ' + filename)
bu.error(msg_str + "Error saving replay: " + filename)
ERROR_N += 1
else:
bu.error(msg_str + 'Replay file is not valid: ' + filename)
bu.error(msg_str + "Replay file is not valid/complete: " + filename)
ERROR_N += 1
except Exception as err:
bu.error(msg_str + 'Unexpected Exception: ' + str(type(err)) + ' : ' + str(err) )
bu.debug(msg_str + 'Marking task done')
bu.error(
msg_str + "Unexpected Exception: " + str(type(err)) + " : " + str(err)
)
bu.debug(msg_str + "Marking task done")
queue.task_done()
return None
def getTitle_old(replayfile: str, title: str, i : int) -> str:
# def getTitle_old(replayfile: str, title: str, i: int) -> str:
# global wg
# if title is None:
# try:
# filename = os.path.basename(replayfile)
# bu.debug(filename)
# map_usrStrs = wg.get_map_user_strs()
# tank_userStrs = wg.get_tank_user_strs()
# # p = re.compile('\\d{8}_\\d{4}_(.+)_(' + '|'.join(map_usrStrs) + ')(?:-\\d)?\\.wotbreplay$')
# # update 6.2 changed the file name format. Bug fixed 2019-09-09 Jylpah
# p = re.compile(
# "\\d{8}_\\d{4}_.*?("
# + "|".join(tank_userStrs)
# + ")_("
# + "|".join(map_usrStrs)
# + ")(?:-\\d)?\\.wotbreplay$"
# )
# m = p.match(filename)
# if m is not None:
# if wg.tanks is not None:
# tank = m.group(1)
# if tank in wg.tanks["userStr"]:
# tank = wg.tanks["userStr"][tank]
# else:
# bu.error(
# 'Tank code: "'
# + tank
# + '" not found from Tankopedia (tanks.json)'
# )
# else:
# tank = m.group(1)
# blitz_map = m.group(2)
# if blitz_map in wg.maps:
# blitz_map = wg.maps[blitz_map]
# else:
# bu.error(
# 'Mapcode: "'
# + blitz_map
# + '" not found from map database (maps.json)'
# )
# title = tank + " @ " + blitz_map
# else:
# title = re.sub("\\.wotbreplay$", "", filename)
# except Exception as err:
# bu.error(err)
# else:
# title.replace("NN", str(i))
# return title
def getTitle(replayfile: str, title: str | None = None, i: int = 0) -> str:
global wg
assert wg is not None, "wg must not be None"
if title == None:
try:
filename = os.path.basename(replayfile)
bu.debug(filename)
map_usrStrs = wg.get_map_user_strs()
tank_userStrs = wg.get_tank_user_strs()
#p = re.compile('\\d{8}_\\d{4}_(.+)_(' + '|'.join(map_usrStrs) + ')(?:-\\d)?\\.wotbreplay$')
# update 6.2 changed the file name format. Bug fixed 2019-09-09 Jylpah
p = re.compile('\\d{8}_\\d{4}_.*?(' + '|'.join(tank_userStrs) + ')_(' + '|'.join(map_usrStrs) + ')(?:-\\d)?\\.wotbreplay$')
m = p.match(filename)
if (m != None):
if wg.tanks != None:
tank = m.group(1)
if tank in wg.tanks['userStr']:
tank = wg.tanks['userStr'][tank]
else:
bu.error('Tank code: "' + tank + '" not found from Tankopedia (tanks.json)')
else:
tank = m.group(1)
blitz_map = m.group(2)
if blitz_map in wg.maps:
blitz_map = wg.maps[blitz_map]
else:
bu.error('Mapcode: "' + blitz_map + '" not found from map database (maps.json)')
title = tank + ' @ ' + blitz_map
else:
title = re.sub('\\.wotbreplay$', '', filename)
except Exception as err:
bu.error(err)
else:
title.replace('NN', str(i))
return title
def getTitle(replayfile: str, title: str = None, i : int = 0) -> str:
global wg
if title == None:
if title is None:
try:
filename = os.path.basename(replayfile)
bu.debug(filename)
tank = None
map_name = None
with zipfile.ZipFile(replayfile, 'r') as archive:
with zipfile.ZipFile(replayfile, "r") as archive:
# bu.debug('Replay file: ' + replayfile + ' opened')
with io.TextIOWrapper(archive.open('meta.json')) as meta:
with io.TextIOWrapper(archive.open("meta.json")) as meta:
# bu.debug('Replay file\'s metadata: opened')
try:
metadata_json = json.load(meta)
#player = metadata_json['playerName']
tank = wg.tank_str2name(metadata_json['playerVehicleName'])
map_name = wg.get_map(metadata_json['mapName'])
bu.debug('Tank: ' + tank + ' Map: ' + map_name)
# player = metadata_json['playerName']
tank_name = wg.get_tank_name(
metadata_json["vehicleCompDescriptor"]
)
map_name = wg.get_map(metadata_json["mapName"])
bu.debug(
"Tank: " + tank_name
if tank_name is not None
else "NOT FOUND IN tanks.json" + " Map: " + map_name
)
except Exception as err:
bu.error(exception = err)
if (tank != None) and (map_name != None):
title = tank + ' @ ' + map_name
bu.error(exception=err)
if (tank_name is not None) and (map_name is not None):
title = tank_name + " @ " + map_name
else:
title = re.sub('\\.wotbreplay$', '', filename)
title = re.sub("\\.wotbreplay$", "", filename)
except Exception as err:
bu.error(err)
else:
title.replace('NN', str(i))
bu.debug('Returning: ' + title)
title.replace("NN", str(i))
bu.debug("Returning: " + title)
return title
### UNIT TESTS - pytest
# class TestClass:
# def test_getTitle(self):
# #player = "jylpah"
# WG_appID = '81381d3f45fa4aa75b78a7198eb216ad'
# wg = WG(WG_appID, 'tanks.json', 'maps.json')
# tank_name = "T28 Defender" # "A68_T28D"
# map_name = "Yukon" # "idle"
# replay = './test/' + '20201030_2358__jylpah_A_1157485188135816346.wotbreplay'
# res = tank_name + ' @ ' + map_name
# def test_getTitle(self):
# #player = "jylpah"
# WG_appID = '81381d3f45fa4aa75b78a7198eb216ad'
# wg = WG(WG_appID, 'tanks.json', 'maps.json')
# tank_name = "T28 Defender" # "A68_T28D"
# map_name = "Yukon" # "idle"
# replay = './test/' + '20201030_2358__jylpah_A_1157485188135816346.wotbreplay'
# res = tank_name + ' @ ' + map_name
# assert getTitle(replay) == res
# assert getTitle(replay) == res
# @pytest.mark.asyncio
# async def test_getTitle(self):
# #player = "jylpah"
# WG_appID = '81381d3f45fa4aa75b78a7198eb216ad'
# wg = WG(WG_appID, 'tanks.json', 'maps.json')
# tank_name = "T28 Defender" # "A68_T28D"
# map_name = "Yukon" # "idle"
# replay = './test/' + '20201030_2358__jylpah_A_1157485188135816346.wotbreplay'
# res = tank_name + ' @ ' + map_name
# @pytest.mark.asyncio
# async def test_getTitle(self):
# #player = "jylpah"
# WG_appID = '81381d3f45fa4aa75b78a7198eb216ad'
# wg = WG(WG_appID, 'tanks.json', 'maps.json')
# tank_name = "T28 Defender" # "A68_T28D"
# map_name = "Yukon" # "idle"
# replay = './test/' + '20201030_2358__jylpah_A_1157485188135816346.wotbreplay'
# res = tank_name + ' @ ' + map_name
# assert getTitle(replay) == res
# assert getTitle(replay) == res
### main() -------------------------------------------
if __name__ == "__main__":
@@ -338,6 +438,8 @@ if __name__ == "__main__":
except ImportError:
pass
else:
if not isinstance(asyncio.get_event_loop_policy(), WindowsSelectorEventLoopPolicy):
if not isinstance(
asyncio.get_event_loop_policy(), WindowsSelectorEventLoopPolicy
):
asyncio.set_event_loop_policy(WindowsSelectorEventLoopPolicy())
asyncio.run(main(sys.argv[1:]), debug=False)