depot - monorepo for the virus lounge

Age	Commit message (Collapse)	Author	Files	Lines
2023-01-25	r/5755 fix(corp/data-import): `rank` is an integer field	Vincent Ambo	2	-2/+2
	Change-Id: Ifc9cd46e5b5521096db19628bd8bcf026106dcc9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7926 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-24	r/5752 feat(corp/data-import): map OR word types to sets of OC grammemes	Vincent Ambo	1	-0/+13
	Change-Id: I674f3a66fcd65314431a2ebd747e3830aa2dd7a1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7924 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su>
2023-01-24	r/5751 feat(corp/data-import): map OC lemma grammemes to OR form types	Vincent Ambo	1	-15/+103
	Change-Id: Ie804d185269336b0d9fe417754e5e795918e65b8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7923 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-24	r/5750 feat(corp/data-import): map OC word grammemes to OR form types	Vincent Ambo	2	-0/+85
	This table maps the grammemes for individual word forms (not for lemmata in either corpus!) to the corresponding grammemes from the other dataset. These have drastically different shapes, so the mapping is not perfect, but will help in determining which forms are intended to be the same on both sides. Change-Id: Ib0717e2f7a79d96bcb5e955a20f551e391fcd759 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7918 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2023-01-22	r/5732 feat(corp/data-import): add import of OR 'translations' table	Vincent Ambo	3	-0/+70
	The original dataset contains translations into different languages, but only the English ones are imported here. Note that translations are for lemmata only. Change-Id: Ifb9c32c25fda44c38ad899efca9d205c520c0fa3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7895 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-21	r/5730 feat(corp/data-import): add import of OR 'words_forms' table	Vincent Ambo	3	-6/+69
	This is the full morphological set table for all the words from the lemmata table, which they don't call it that. Change-Id: I6f5be673c5f59f11e36bd8c8c935844a7d4fd170 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7894 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-21	r/5729 feat(corp/data-import): add import of OpenRussian 'words' table	Vincent Ambo	3	-27/+223
	This is actually the lemmata table of this corpus, not the forms of all words (they're in a separate table). Change-Id: I89a2c2817ccce840f47406fa2a636f4ed3f49154 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7893 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-18	r/5703 docs(corp/data-import): document OpenRussian format	Vincent Ambo	1	-4/+53
	This is the second dataset I want to integrate as it contains some more practically useful, but somewhat less structured, information. Change-Id: Ib46b2597a33e76f59e030f889a0961ecc5a144eb Reviewed-on: https://cl.tvl.fyi/c/depot/+/7873 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-18	r/5702 chore(corp/data-import): namespace tables for OpenCorpora data	Vincent Ambo	2	-22/+22
	I'm changing strategies to importing both OC and another dataset before continuing to normalise the data, as it might be easier to do in a set of table-constructing queries inside of SQLite with all raw data in place. Change-Id: I26b41af80586fc1bfd8e26a6be20579068a82507 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7872 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-18	r/5692 feat(corp/data-import): let users specify output path	Vincent Ambo	1	-6/+14
	Change-Id: I61ad021c7a5318b099f3adc8bc6aedef65500974 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7865 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-18	r/5691 feat(corp/data-import): parse and import links	Vincent Ambo	2	-3/+78
	Change-Id: Iebdbc8f884f28064d7b00b8f8808b5030fa3d05c Reviewed-on: https://cl.tvl.fyi/c/depot/+/7864 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-18	r/5690 feat(corp/data-import): parse and import link types	Vincent Ambo	2	-2/+54
	Change-Id: Iae01d1dc6894117dc693b4690d8bc79861212ae6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7863 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-18	r/5689 fix(corp/data-import): commit the final transaction, too	Vincent Ambo	1	-0/+2
	Otherwise up to 1000 elements might be missing. Change-Id: I20d6238424eec27f0e758e7737c9c31bcb81b23d Reviewed-on: https://cl.tvl.fyi/c/depot/+/7862 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-18	r/5688 feat(corp/data-import): insert OpenCorpora data into SQLite	Vincent Ambo	2	-9/+155
	This is an initial and kind of dumb table structure, but there's some massaging that needs to be done before this makes more sense. Change-Id: I441288b684ef86be507099bcc4ebf984598789c8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7861 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-18	r/5684 feat(corp/data-import): parse lemmas from OpenCorpora dump	Vincent Ambo	2	-14/+135
	Change-Id: I1e4efcfc8e555f61578b563411d5e6ed9590d8e8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7860 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-18	r/5683 feat(corp/russian/data-import): new OpenCorpora data import tool	Vincent Ambo	2	-0/+388
	Adds the beginning of a tool which can import OpenCorpora data into a SQLite database. This is quite a lot of toil and there's probably a better way to do this, but overall becoming this intimately familiar with the data structures is quite helpful for understanding what I can/can't do with only this dataset. Change-Id: Ieab33a8ce07ea4ac87917b9c8132226bbc6523b1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7859 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI