about summary refs log tree commit diff
path: root/third_party/lisp
AgeCommit message (Collapse)AuthorFilesLines
2024-12-04 r/8983 feat(3p/lisp/mime4cl): add benchmark scriptsterni4-1/+100
This is far from comprehensive, mainly covering stuff I'm interested for mblog currently. I should extend it as I go. The cases I've added reveal something I've noticed recently: The worst performing part of mime4cl seems to be the initial parsing of the message. My current theory is that this is due to the use of READ-LINE in DO-MULTIPART-PARTS which seems to ultimately dispatch to READ-CHAR internally due to the way our streams are set up. We should look into fixing this soon. It may be interesting to add this to windtunnel at some point, but I'd rather not burden a runner with this given that mime4cl is only worked on once every blue moon and I'm the only user. Change-Id: I001de3aac01f8aa7ea923b43b2db29cf66a4aac3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12864 Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2024-12-02 r/8980 chore(3p/lisp/closure-*): disable on ECL for nowsterni2-1/+10
As it turns out, some of the load/compile time set up the package does doesn't work in ECL for unknown reasons at the moment. Executables using closure-* will crash after starting up: ;;; Checking for wide character support... WARNING: Lisp implementation doesn't use UTF-16, but accepts surrogate code points. yes, using code points. ;;; Building Closure with CHARACTER RUNES Condition of type: SIMPLE-ERROR Invalid relative pathname #P"package.lisp" for component ("closure-common" "package") Change-Id: I4b4bf96835a39696884ec6fea9c249fdeb53c853 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12863 Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2024-12-02 r/8979 feat(3p/lisp/mime4cl): enable compilation with CCLsterni3-5/+3
Only significant implementation specific code at the moment is FILE-SIZE which isn't very important. We can also easily implement it for CCL. Additionally, we clean up an unused lexical variable warning and remove a duplicate definiton of MIME-TYPE-STRING fro MIME-UNKNOWN-PART that CCL doesn't like. Change-Id: I7c960e50dcdc1d3e46cb4945f36ea315a3c9838d Reviewed-on: https://cl.tvl.fyi/c/depot/+/12862 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2024-12-02 r/8978 fix(3p/lisp/mime4cl): fix stat function name in FILE-SIZEsterni1-2/+1
This is a silly mistake and was not caught because FILE-SIZE isn't exercised in the test suite. We can probably remove MIME-BODY-SIZE and look into removing MIME-PART-SIZE as well. I just want to be thorough here so that we can revert into a non-broken state in case we decide we need those functions for something. Change-Id: I5bbb3dde6616220fc3b6feddbf7a39b6a9b0ea0d Reviewed-on: https://cl.tvl.fyi/c/depot/+/12861 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2024-12-02 r/8977 refactor(3p/lisp/mime4cl): drop NATIVE-NAMESTRINGsterni1-5/+0
The only code that used this function was removed in r/7854. Change-Id: Ia07dcb08ed4a92495085b48018372fb9898a0248 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12860 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2024-12-02 r/8975 fix(3p/lisp/mime4cl): don't store redundant headers in MIME-MESSAGEsterni2-7/+230
MIME-MESSAGE has a HEADERS slot which is an alist of all headers. Some of those headers will be parsed again and stored in MIME-PART (or a subclass of it). Having the header content stored in the HEADERS alist and in MIME-PART causes problems: - Requires extra knowledge about how messages are parsed when rendering messages. - Makes MIME= depend on the specific whitespace and quoting in those headers which isn't preserved by how mime4cl parses e.g. Content-Type. - Gives users two ways that slightly diverge to access the same thing. To avoid this, we remove these headers after the MIME-PARTs contained in MIME-MESSAGE have been initialized (since they reuse the HEADERS slot). Change-Id: I5b221f88bbac47dd81db369e3c1d5881a5a50e5e Reviewed-on: https://cl.tvl.fyi/c/depot/+/12858 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: sterni <sternenseemann@systemli.org>
2024-12-02 r/8974 fix(3p/lisp/mime4cl): also default to :7BIT when decodingsterni1-5/+7
Content-Transfer-Encoding should default to 7bit when it's not given (RFC2045). MIME-PART already defaults to this when manually constructing this, but MAKE-MIME-PART would always set it, so it would sometimes be NIL which is incorrect. We now correctly fall back to :7bit in this case. Additionally, we make sure that KEYWORDIFY-ENCODING immediately returns when it's given NIL. Change-Id: I50f86dd649d83a4c3a8881d6e13dcada889d5521 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12857 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org>
2024-12-02 r/8973 refactor(3p/lisp/mime4cl): streamline MIME-MESSAGE dispatchingsterni1-3/+3
Change-Id: I1fda161e6e128f1bb085a63dd39541894e397d64 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12856 Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2024-12-02 r/8972 fix(3p/lisp/mime4cl): support FILE-PORTION in PRINT-MIME-PARTsterni1-0/+2
Change-Id: I942e8915d5076628179dfa77bf80b7510b862b51 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12855 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2024-12-02 r/8971 refactor(3p/lisp/mime4cl): eliminate use of READ-CHAR in PRINT-MIME-PARTsterni1-4/+1
Change-Id: Ibb422d3b6720b782620e262bbd3555b9a879ad65 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12854 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: sterni <sternenseemann@systemli.org>
2024-04-05 r/7854 refactor(3p/mime4cl): use SB-POSIX for FILE-LENGTHsterni2-43/+5
This is slightly better than the (mostly untested) mess we had before: Just implement the one thing we need using the tools the one implementation we support (SBCL) gives us. Eventually, we'll want to make this portable, probably using osicat. Unfortunately, packaging this requires support for cffi-grovel (b/383) which buildLisp lacks at the moment. Change-Id: I6960015f80e6a5dfde67baf55537c5274a19e4e2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11356 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org>
2024-03-31 r/7825 feat(3p/lisp): Add str package and dependenciesAspen Smith3-0/+83
Change-Id: Iffb66f4580517b1dbfee8c79e766552508695e5f Reviewed-on: https://cl.tvl.fyi/c/depot/+/11252 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Autosubmit: aspen <root@gws.fyi>
2023-12-20 r/7231 chore(depot): update OWNERS files for aspenLuke Granger-Brown1-1/+1
Change-Id: Id94b646a6ea035782298c421d6667530da6fc5b6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10384 Tested-by: BuildkiteCI Owners-Override: lukegb <lukegb@tvl.fyi> Reviewed-by: lukegb <lukegb@tvl.fyi>
2023-05-23 r/6182 docs(3p/lisp/mime4cl): describe changes compared to original versionsterni2-7/+27
Spell out that “may diverge” is more of a “has diverged by now”. We are essentially maintaining a fork of mime4cl. Change-Id: I9049e8296a666c3d1b08eae28813147f360771ef Reviewed-on: https://cl.tvl.fyi/c/depot/+/8621 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2023-05-18 r/6158 test(3p/lisp/mime4cl): test decoding RFC2047 examplessterni3-2/+22
Change-Id: I32abb00e8cec697adb45b9a175cd753e807d33d5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8588 Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2023-05-18 r/6157 refactor(3p/lisp/mime4cl): remove unused DECODE-STREAMsterni1-11/+0
This function has apparently been unused ever since we imported mime4cl into depot. Change-Id: I224c9b2947ffd641e01e8cd5454d7a7fa75278d4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8587 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org>
2023-05-18 r/6156 refactor(3p/lisp/mime4cl): port remaining base64 decoding to qbase64sterni4-75/+39
DECODE-BASE64-STREAM-TO-SEQUENCE is the only thing that requires anything fancy: We read into an adjustable array. Alternative could be using REDIRECT-STREAM and WITH-OUTPUT-TO-STRING, but that is likely slower (untested). Test cases are kept for now to confirm that qbase64 is conforming to our expectations, but can probably dropped in favor of a few more sample messages in the test suite. :START and :END are sadly no longer supported and need to be replaced by SUBSEQ. Change-Id: I5928aed7551b0dea32ee09518ea6f604b40c2863 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8586 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org>
2023-05-18 r/6155 refactor(3p/lisp/mime4cl): remove be and be*sterni6-117/+94
Seems simple enough to use standard LET and a few parentheses more which stock emacs can indent probably. Change-Id: I0137a532186194f62f3a36f9bf05630af1afcdae Reviewed-on: https://cl.tvl.fyi/c/depot/+/8584 Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2023-05-18 r/6154 refactor(sterni/mblog): move REDIRECT-STREAM into mime4clsterni2-9/+20
Eventually, we'll want to replace dump-stream-binary with something more efficient—given that we have flexi-streams we can use something that only does matching element types no problem. REDIRECT-STREAM is much more efficient thanks to using an internal buffer. streams.lisp gets a new section at the beginning for grouping utilities that don't have any real (internal) dependencies. Change-Id: I141cd36440d532131f389be2768fdaa54e7c7218 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8583 Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2023-05-18 r/6153 refactor(3p/lisp/mime4cl): use qbase64 for decoding FILE-PORTIONssterni2-2/+2
Porting over the rest of the decoding (RFC2047) and especially encoding over to qbase64 is still pending, as it is a little trickier. Change-Id: Id4740eb074a387aeea2cb94b781e204248530799 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8582 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2023-05-18 r/6152 refactor(mime4cl): replace *-input-adapter-stream with flexi-streamssterni2-124/+58
The input adapter streams were input streams yielding either binary or character data that could be constructed from a variable data source. The stream would take care not to destroy the underlying data source (i.e. not close it if it was a stream), so similar to with FILE-PORTIONs, but simpler. Unfortunately, the implementation was quite inefficient: They are ultimately defined in terms of a function that retrieves the next character in the source. This only allows for an implementation of READ-CHAR (and READ-BYTE). Thanks to cl/8559, READ-SEQUENCE can be used on e.g. FILE-PORTION, but this was still negated by a input adapter based on one—then, READ-SEQUENCE would need to fall back on READ-CHAR or READ-BYTE again. Luckily, we can replace BINARY-INPUT-ADAPTER-STREAM and CHARACTER-INPUT-ADAPTER-STREAM with a much simpler abstraction: Instead of extra stream classes, we have a function, MAKE-INPUT-ADAPTER, which returns an appropriate instance of FLEXI-STREAM based on a given source. This way, the need for a distinction between binary and character input adapter is eliminated, since FLEXI-STREAMS supports both binary and character reads (external format is not yet handled, though). Consequently, the :binary keyword argument to MIME-BODY-STREAM can be dropped. flexi-streams provides stream classes for everything except a stream that doesn't close the underlying one. Since we have already implemented this in POSITIONED-FLEXI-INPUT-STREAM, we can split this functionality into a new superclass ADAPTER-FLEXI-INPUT-STREAM. This change also allows addressing the performance regression encountered in cl/8559: It seems that flexi-streams performs worse when we are reading byte by byte or char by char. (After this change mblog is still two times slower than on r/6150.) By eliminating the adapter streams, we can start utilizing READ-SEQUENCE via decoding code that supports it (i.e. qbase64) and bring performance on par with r/6150 again. Surely there are also ways to gain back even more performance which has to be determined using profiling. Buffering more aggressively seems like a sure bet, though. Switching to flexi-streams still seems like a no-brainer, as it allows us to drop a lot of code that was quite hacky (e.g. DELIMITED-INPUT- STREAM) and implements en/decoding handling we did not support before, but would need for improved correctness. Change-Id: Ie2d1f4e42b47512a5660a1ccc0deeec2bff9788d Reviewed-on: https://cl.tvl.fyi/c/depot/+/8581 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2023-05-18 r/6151 refactor(3p/lisp/mime4cl): use flexi-streams and binary inputsterni2-129/+115
This refactor is driven by the following (ultimate) aims: - Get rid of as much of the custom stream code in mime4cl which makes less code to maintain in the future. - Lay the groundwork for correct handling of 8bit transfer encoding: The mime4cl we inherited assumes that any MIME message can be decoded completely by the CL implementation (in SBCL's case using latin1) into CHARACTERs. This is not necessarily the case. flexi-streams allows changing how the stream is decoded on the fly and also has support for reading the underlying bytes which is perfect for the requirements decoding MIME has. - Since flexi-streams uses trivial-gray-streams, it supports READ-SEQUENCE. Taking advantage of this may improve decoding performance significantly in the future. This incurs the following changes: - Naturally we now open given files as binary files in MIME-MESSAGE. Given strings are encoded using STRING-TO-OCTETS and then passed on to a new octet vector method. Instead of MY-STRING-INPUT-STREAM this now uses flexi-streams' WITH-INPUT-FROM-SEQUENCE. - OPEN-FILE-PORTION and OPEN-DECODED-FILE-PORTION need to be merged, since the transfer encoding not only implies an extra decoder stream that needs to be attached after file portion stream, but also imply a certain encoding of the stream itself (mostly binary vs. ASCII). As flexi-streams can change their encoding on the fly this could be untangled again, but it is not strictly necessary. As before, we use the DATA slot of the file portion to create a fresh stream if possible. Instead of strings we now use an vector of octets to match MIME-MESSAGE. The actual portioned stream relies on POSITIONED-FLEXI-INPUT-STREAM, a subclass of the stock FLEXI-INPUT-STREAM class, described below. - POSITIONED-FLEXI-INPUT-STREAM replaces DELIMITED-INPUT-STREAM. It is created using MAKE-POSITIONED-FLEXI-INPUT-STREAM which accepts the same arguments as MAKE-FLEXI-STREAMS and, additionally, :IGNORE-CLOSE. A POSITIONED-FLEXI-INPUT-STREAM works the same as an FLEXI-INPUT-STREAM, but upon creation, the underlying stream is rewinded or forwarded to the argument given by :POSITION using FILE-POSITION. If :IGNORE-CLOSE is T, a call to CLOSE is not forwarded to the underlying stream. Change-Id: I2d48c769bb110ca0b7cf52441bd63c1e1c2ccd04 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8559 Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2023-05-16 r/6147 refactor(3p/lisp/mime4cl/test): create one test case per sample filesterni2-22/+24
Since rt.lisp seems to start tests in parallel, the informational output about which sample file is being tested gets mangled in all sorts of ways. The solution is to just loop over the sample files outside a test and schedule a single test case per sample file from there. Change-Id: I4494e4a526ce6d92a298cf7daf06c8013c7ca605 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8569 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2023-05-14 r/6137 feat(3p/lisp): add qbase64sterni2-0/+70
Change-Id: I448b9241726c3bb08f14188775a66e1da1225e02 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5004 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: sterni <sternenseemann@systemli.org>
2023-05-09 r/6128 fix(3p/lisp/mime4cl): use OTHERWISE in CASE not Tsterni1-1/+1
Change-Id: Ia674705b27fbc4ae3055973eec563b078a4a873c Reviewed-on: https://cl.tvl.fyi/c/depot/+/8558 Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2023-05-09 r/6127 fix(3p/lisp/mime4cl/tests): fix sample discovery in nix buildsterni1-1/+1
CL's path handling strikes once again… Change-Id: I4345941c8e2856f80cfddecc5356464f92b1a150 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8557 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: sterni <sternenseemann@systemli.org>
2023-05-09 r/6126 refactor(3p/lisp/mime4cl): drop unused split-multipart-partssterni1-28/+0
Change-Id: If47a8ffde5b4910f6c52fe82a2372431a0e46045 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8556 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2023-05-09 r/6125 refactor(3p/lisp/mime4cl): rename :stream to :underlying-streamsterni2-11/+12
This makes sure that initializing coder-stream-mixin (for the most part) has the same interface as initializing qbase64:decode-stream. This will make integrating that as a faster replacement to mime4cl:base64-decoder-stream a bit easier. The idea is to replace the char by char base64 decoder with one that supports read-sequence. After that deliminited-input-stream needs to gain support for read-sequence as well, so we can actually take advantage of this fact. Finally, we'll have to evaluate the remaining decoders and think about switching the (base64) encoders over as well. Change-Id: If971da02437506e00a7c9fab2b94efc42725e62d Reviewed-on: https://cl.tvl.fyi/c/depot/+/8555 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org>
2023-05-01 r/6122 refactor(3p/lisp/mime4cl): unify test mechanism for sample msgssterni3-17/+4
For whatever reason, there were two sort of identical tests, mime.1 and mime.2, in the mime4cl test suite: The former tested *sample1-file* and the latter all messages *samples-directory*—in the same way, parsing the original and a re-rendered version of the message to check if they were equal. We can just move sample1.msg into *samples-directory*, get rid of *sample1-file* and thus pave the way for more test messages in the future. Change-Id: I843be331682b731af6ae02a4648ba1c64aaf59a5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8546 Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2023-04-29 r/6119 fix(3p/lisp/mime4cl): correctly define find-mime-text-partsterni1-1/+1
The generic function itself needs to be defined using defgeneric, defmethod is used for a defining method of a generic function, i.e. how it should behave when confronted with a certain class. Change-Id: Idd38afa02b56c5002e215decfff7f0c25267eab5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8532 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2023-03-30 r/6063 refactor(3p/lisp/mime4cl): replace babel with flexi-streamssterni3-7/+5
decode-RFC2047 used babel's octets-to-string, but we can replace it with the function of the same name from flexi-streams. This doesn't make a difference for the moment, but will be useful in the future: flexi-streams provides de- and encoding streams that we'll be able to use to replace and augment some of the stream based MIME part handling code in mime4cl. babel doesn't have as powerful stream functionality although it seems to be planned. Another big upside of flexi-streams is that we'll be able to replace delimited-input-string using it. This should allow us to slowly work towards correct and more efficient decoding of MIME bodies. Change-Id: I17174f1c96c5be7d103d396564e6aa0fe24c80fc Reviewed-on: https://cl.tvl.fyi/c/depot/+/8371 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-12-19 r/5427 feat(3p/lisp/lisp-binary): 2022-04-10 -> 2022-09-19sterni1-10/+6
Add missing dependency alexandria. This update adds a feature to disable the cffi which would be neat for ECL. Change-Id: Iad5a4646317fb26bb2dec7bcf3d883075ab24842 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7564 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-09-19 r/4922 chore(gerrit): migrate OWNERS files to code-owners styleLuke Granger-Brown3-11/+4
Change-Id: Iacc521dfdd4b4a2d5cef3920cf8189bcce35a488
2022-08-30 r/4546 feat(nix/buildLisp): re-enable CCLsterni1-2/+2
The problem went away once again, let's see how long it'll last this time. As it turns out, CCL has a Unicode Standard conforming string implementation that doesn't allow the use of (lone) surrogate code points, requiring us to disable a test in cl-json which tested the behavior of en- and decoding of such a (technically illegal) string. Change-Id: I8bfa482934bbf94f86cecdde02d5c3d4e77950a5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6204 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su>
2022-07-05 r/4278 fix(3p/lisp/cl-json): en/decode non-BMP characters correctlysterni1-2/+31
Pin cl-json to a branch on my fork of cl-json which implements encoding and decoding of non-BMP Unicode codepoints (>= U+10000) using UTF-16 surrogate points as the JSON specs prescribes. While we're at it, also enable the test suite of the package. This resolves the annoying issue of panettone garbling up some Unicode characters by sending invalid JSON to cheddar and then incorrectly decoding the returned result. Fixes: b/145. Change-Id: I44cb38c2775c0e5512d75f51dff0d1deb39f8f78 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5884 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su>
2022-07-05 r/4275 chore: remove sclf from the treesterni22-3645/+472
SCLF is quite a big utility library (almost 3€ LOC) with limited portability (CMUCL, SBCL and CLISP to an extent). Continuing to maintain it is an unnecessary burden, as depot only uses a fraction of it which is now inlined into the respective users (mime4cl and mblog). In the future trimming down ex-sclf.lisp may make sense either by refactoring the code that uses it or by moving interesting utilities into e.g. klatre. Change-Id: I2e73825b6bfa372e97847f25c30731a5aad4a1b5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5922 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: sterni <sternenseemann@systemli.org>
2022-05-31 r/4193 chore(3p/lisp/cl-json): use quicklisp sourcesterni1-6/+1
This switches upstream from hankhero/cl-json to sharplispers/cl-json (the former of which had its last commit in 2014). Sadly the new upstream hasn't decided on an appropriate fix for b/145 yet (due to concern about backwards compatibility, apparently). I did not look before working on a fix, so I have an 90% finished fix which is (I think) better than the already proposed ones, so I'll patch it in here eventually. Change-Id: I9e39e138fa655794b864db5f268bdfdc35788fcc Reviewed-on: https://cl.tvl.fyi/c/depot/+/5795 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-02-02 r/3754 feat: move mblog header handling into mime4clsterni4-2/+34
Accessing the headers of a MIME message feels like something mime4cl should handle. We implemented this ad hoc in mblog before in order to not need to worry about doing it in a sensible way. Now we introduce a decent-ish interface for getting a header from a MIME message, mime-message-header-values: * It returns a list because MIME message headers may appear multiple times. * It decodes RFC2047 only upon request, as you may want to be stricter about parsing certain fields. * It checks header name equality case insensitively. The code for decoding the RFC2047 string is retained and still uses babel for doing the actual decoding. Change-Id: I58bbbe4b46dbded04160b481a28a40d14775673d Reviewed-on: https://cl.tvl.fyi/c/depot/+/5150 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-02 r/3749 feat(3p/lisp/mime4cl): cache offset in delimited-input-streamsterni1-19/+39
By computing the amount the stream position advanced we can save a syscall on every read which speeds up mime:mime-body-stream by /a lot/, e.g. extracting a ~3MB attachment drops from over 15s to under ~0.5s. There's still a lot to be gained and correctness left to be desired which can be addressed as described in the newly added comment. Change-Id: I5e1dfd213aac41203f271cf220db456dfb95a02b Reviewed-on: https://cl.tvl.fyi/c/depot/+/5073 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-01-31 r/3723 style: format entire depot with nixpkgs-fmtVincent Ambo25-66/+91
This CL can be used to compare the style of nixpkgs-fmt against other formatters (nixpkgs, alejandra). Change-Id: I87c6abff6bcb546b02ead15ad0405f81e01b6d9e Reviewed-on: https://cl.tvl.fyi/c/depot/+/4397 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: lukegb <lukegb@tvl.fyi> Reviewed-by: wpcarro <wpcarro@gmail.com> Reviewed-by: Profpatsch <mail@profpatsch.de> Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: cynthia <cynthia@tvl.fyi> Reviewed-by: edef <edef@edef.eu> Reviewed-by: eta <tvl@eta.st> Reviewed-by: grfn <grfn@gws.fyi>
2022-01-28 r/3692 feat(3p/overlays/ecl-static): 21.2.1 -> 1c98924sterni1-4/+0
Seems like some issues to do with bytecode compilation have been fixed at HEAD. closer-mop compiles again and an ironclad failure with the next quicklisp/channel bump is avoided. In this change pathname handling in ECL also changed somehow, causing it to make the :directory part absolute by prefixing it with a slash which made ld.bfd unhappy while linking an output path that began with a double slash. This problem can be avoided by constructing the path as ANSI Common Lisp intended. The truename on the out path is important to make it recognize that it is indeed a directory. Change-Id: I5e744022b92502f99ac0b33411a6be443707e200 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5076 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-01-26 r/3677 chore(3p/lisp/mime4cl): remove CMUCL specific codesterni7-18/+11
Having #+cmu all over the place suggests that we maintain CMUCL support or test with CMUCL which is not the case. Change-Id: Ia0828cb1ac48e49acdee6fef7a0fa2c04c1805b3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5068 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-01-26 r/3676 refactor(3p/lisp/mime4cl): use trivial-gray-streamssterni4-51/+16
This should be a net positive for portability and lets us drop some of the CMUCL cruft (which we don't test anyway, CMU support may have regressed regardless). Change-Id: I85664d82d211177da1db9eebea65c956295b09f7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5067 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-01-26 r/3675 style(3p/lisp): expand tabs in npg, mime4cl and sclfsterni25-2467/+2467
Done using find third_party/lisp/{sclf,mime4cl,npg} \ -name '*.lisp' -or -name '*.asd' \ -exec bash -c 'expand -i -t 8 "$0" | sponge "$0"' {} \; Change-Id: If84afac9c1d5cbc74e137a5aa0ae61472f0f1e90 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5066 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-01-26 r/3674 feat(3p/lisp/cl-date-time-parser): init at 00d6fc7sterni1-0/+21
Change-Id: I6a04c5bb611f011e66c1a8d7089e0b2bff3d2fdf Reviewed-on: https://cl.tvl.fyi/c/depot/+/5002 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su>
2022-01-19 r/3635 feat(3p/lisp): add metabang-bindsterni1-0/+16
Change-Id: I6efcb07b7763c4d108baff717d07097cb324ba4f Reviewed-on: https://cl.tvl.fyi/c/depot/+/5003 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: sterni <sternenseemann@systemli.org>
2022-01-17 r/3619 fix(3p): Move away from builtins.fetchGitVincent Ambo1-1/+2
Moves to the derivation-based git fetchers everywhere in third-party. This might help with forward-compatibility with newer Nix versions, though that's not our primary concern right now. Change-Id: I565bb72585b8639893e9ea3a9e233338aede63a9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3903 Tested-by: BuildkiteCI Reviewed-by: zseri <zseri.devel@ytrizja.de>
2022-01-05 r/3520 chore(3p/lisp): remove unused lisp packagesVincent Ambo4-110/+0
Change-Id: Id259341e251170c1caeeab5c9fcb6fbd973372f8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4816 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-12-25 r/3404 chore(3p): bump NixOS channels to 2021-12-25sterni1-0/+4
* 3p/lisp/closer-mop: closer-mop no longer builds with ECL (see linked issue), so let's mark it as broken for now. Change-Id: I97c29d718682cec4ecc682ff1593d0ce9aca0010 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4607 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-12-15 r/3258 chore(3p/lisp): use nixpkgs sources for s-xmlVincent Ambo24-2383/+43
this one was a little more difficult because it needs a patch, there's something wonky with the definition order fwiw, the upstream cvs repository ... server errors. Change-Id: I2d99359edec36b578389f1be1fcf077743c29c4e Reviewed-on: https://cl.tvl.fyi/c/depot/+/4342 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>