![[Home]](http://www.bayleshanks.com/cartoonbayle.png)
https://sneklang.org/doc/snek.html has, among other things, the following curses functions:
16. Curses built-in functions
16.1. curses.initscr()
16.2. curses.endwin()
16.3. curses.noecho(), curses.echo(), curses.cbreak(), curses.nocbreak()
16.4. stdscr.nodelay( nodelay )
16.5. stdscr.erase()
16.6. stdscr.addstr( row , column , string )
16.7. stdscr.move( row , column )
16.8. stdscr.refresh()
16.9. stdscr.getch()---
A NEW LIBRARY FOR SMALL-C by James Hendrix and Ernest Payne
archive10/ddj_devnetwork_small_c from https://www.drdobbs.com/developer-network-small-c-compiler-book/184415519?queryText=%2522small%2Bc%2522 ddj_devnetwork_small_c/articles/chaps/chap6/chap6.htm
---
https://fuchsia.dev/fuchsia-src/reference/syscalls
https://fuchsia.dev/fuchsia-src/reference https://fuchsia.dev/fuchsia-src/concepts
---
https://docs.zephyrproject.org/latest/guides/c_library.html
"
The following functions are implemented in the minimal C library included with Zephyr:
abs()
atoi()
bsearch()
calloc()
free()
gmtime()
gmtime_r()
isalnum()
isalpha()
isdigit()
isgraph()
isprint()
isspace()
isupper()
isxdigit()
localtime()
malloc()
memchr()
memcmp()
memcpy()
memmove()
memset()
mktime()
rand()
realloc()
snprintf()
sprintf()
strcat()
strchr()
strcmp()
strcpy()
strlen()
trncat()
strncmp()
strncpy()
strrchr()
strstr()
strtol()
trtoul()
time()
tolower()
toupper()
vsnprintf()
vsprintf()"---
https://www.evanmiller.org/statistical-shortcomings-in-standard-math-libraries.html
---
rudolfwinestock 2 hours ago [–]
Related: The MGR Window System https://hack.org/mc/mgr/
“MGR provides each client window with: curses-like terminal control functions, graphics primitives such as line and circle drawing; facilities for manipulating bitmaps, fonts, icons, and pop-up menus; commands to reshape and position windows; and a message passing facility enabling client programs to rendezvous and exchange messages. ”
Essentially, each window was a souped up ASCII terminal with extra escape sequences which implemented primitive graphics. It competed with X-Windows on low-end workstations during the 1980s.
Discussion: https://news.ycombinator.com/item?id=18742611
reply
" Client programs may ask to be informed when a change in the window system occurs, such as a reshaped window, a pushed mouse button, or a message sent from another client program. These changes are called events. MGR notifies a client program of an event by sending it an ASCII character string in a format specified by the client program. Existing applications can be integrated into the windowing environment without modification by having MGR imitate keystrokes in response to user defined menu selections or other events. "
" If you want to compare MGR to the X Window System, you might consider the MGR server as something like a combination of an X server + window manager + xterm, but with xterm's Tektronix 4014 graphics terminal emulation and its DEC VT100 emulation in the same window.
A window in MGR doesn't emulate any existing hardware terminal. It has a new set of terminal codes combining text, vector graphics and basic bitmap graphics. See the termcap entry in the distribution.
Compared to X11, MGR has a very small memory footprint and demands very little of the graphics hardware. "
---
i downloaded the mgr tarball (mgrsrc-0.69.tgz ; listed as "Pristine MGR 0.69 source" on https://hack.org/mc/mgr/) and i had some trouble compiling the documentation, but i finally got it to sorta work. I had to first do
sudo apt-get install linuxdoc-tools-text linuxdoc-tools-latex linuxdoc-tools-info
and then
tbl macros doc.1 doc.2 doc.3 doc.4 doc.5 doc.6 tocindex doc.9
| nroff -ms -rI0 > mgr_manual_compiled_sorta_broken.txt |
which is a mangled version of a line from the file doc/groffit in that tgz. The groffit line was: tbl macros doc.1 doc.2 doc.3 doc.4 doc.5 doc.6 tocindex doc.9
| croff/croff -k -c '\fI' '\fP' | groff -Tps -mgs -rI0 > body.ps |
As you can see, i changed groff to nroff, -mgs to -ms (gs and s are macro libraries, one of the packages linuxdoc-tools-text linuxdoc-tools-latex linuxdoc-tools-info seems to give me s), removed the Tps option, and removed the croff because i don't know where to get croff; there is a version of croff in this tgz in doc/croff, but it doesn't immediately compile (i bet it was written for an older version of C!). The result seems pretty readable even though the croff step was removed, however there are a bunch of errors emitted like
doc.1:292: macro error: KE without KS or KF
and also an error about tocindex, which hasn't been built.
I saved mgr_manual_compiled_sorta_broken.txt in my personal archive10 folder.
---
" ...Windows 1.0 GUI...
The bundled apps (including MS-DOS Executive) and the window decorations generally used the same GDI (Graphics Device Interface) calls as third party apps:
Text was drawn with TextOut?() or DrawText?().
Bitmaps were copied to the framebuffer with BitBlt?() or StretchBlt?().
Lines were drawn with MoveTo?() and LineTo?().
Rectangles were drawn with Rectangle() or RoundRect?().
This is not an exhaustive list but should give you the general idea.
All of these functions operated on a "device context" (DC) that you obtained with functions like GetDC?() or CreateCompatibleDC?(). Some, like BitBlt?(), used two DCs for the source and destination.
The MS-DOS Executive drive icons were bitmaps drawn with BitBlt?() with TextOut?() for the drive letter. The selected drive letter and icon were inverted with InvertRect?(), or possibly drawn with the DSTINVERT raster operation code.
These are the same functions that any Windows application could use. The MS-DOS Executive was just another app.
The non-client area of a window (titlebar and such) was drawn with the same GDI calls as the client area. Your app would get a WM_PAINT message to draw the client area and a WM_NCPAINT for the non-client area. Most apps passed WM_NCPAINT through to the default handler DefWindowProc?(). "
---
http://www.sigala.it/sergio/tvision/html/hierarchy.html
---
string.startswith
" How would you add a large standard library without compromising Lua’s main purpose - to be a lightweight, embeddable language?
reply
ebg13 1 day ago [–]
It would still be lightweight and embeddable with common sense pure Lua functionality like classes and major utility methods like string.startswith/string.endswith/string.trim/string.split/table.deepcopy. You don't have to integrate a multiprocessing web server for different operating systems.
Instead you're left with bullshit community essays like http://lua-users.org/wiki/SplitJoin which shows pages and pages of different user attempts for one extremely common function, many of which explicitly do not work! "
fit2rule 1 day ago [–]
1. Luarocks. 2. I regularly carry these little enumerables around with me and use it so frequently I found it quaint to be reminded of the 'hole problem':
--- get all packed entries, until gap
-- func on the provided list
-- @param list table to be inspected
-- @param func do func what you func want
table.each = function(list, func)
for i,v in ipairs(list) do
func(v, i)
end
end --- get all entries, packed or not, until completion
-- func on the provided list, completely
-- @param list table to be inspected
-- @param func do func what you func want
table.all = function(list, func)
if (list ~= nil) then
for i,v in pairs(list) do
func(v, i)
end
end
endDisclaimer: I love Lua and would us it for everything if I could.
reply
ebg13 1 day ago [–]
Do you also carry a table deepcopy with you? What about string.trim?
And LuaRocks? is a trash fire that doesn't integrate well with Lua's primary role as an embedded plugin engine and leaves you to waste your time in endless discovery of a slew of unmaintained packages that all do the same thing in different ways and with different bugs. Want to collaborate? I hope everyone is familiar with "classy" and not "oops" or "objectlua" or "Luaoop" or "lobject" or "classyng" or "klesi" or "halo" or "lua-c3class" or "pool" or "middleclass" or "Sunclass" or any other module that implements one of the half dozen official recommendations for faking classes with metatables or closures. It also becomes a huge "left-pad" risk.
reply
---
https://stevedonovan.github.io/Penlight/api/index.html https://stevedonovan.github.io/Penlight/api/manual/01-introduction.md.html#
---
https://www.google.com/search?channel=fs&client=ubuntu&q=S-Lang+is+like+ncurses slang ncurses
---
LOVE pico8 TIC-80 https://tic80.com/ https://github.com/nesbox/TIC-80/ https://tic80.com/learn https://github.com/nesbox/TIC-80/wiki
tic-80 fork for education: https://github.com/afonsojramos/feup-8/tree/development/Website
https://news.ycombinator.com/item?id=24407370 mentions also Pq93, Pixelvision 8, Liko-12, PX8 but says they are all similar
---
https://github.com/nesbox/TIC-80/wiki
" Console Demo Available Commands
help - Show available commands
vram - Show 16K VRAM layout (0.80)
ram - Show 80K RAM layout
exit - Exit the application
edit - Switches to the code editor
new [lua | moon | js | wren | fennel] - Create new Hello World cartridge
load <cart> [sprites | map | cover | code | sfx | music | palette] - Load cartridge from the local filesystem (there's no need to type the .tic extension). You can also load just the data (sprites, map etc) from another cart
save <cart> - Save cartridge to the local filesystem, use .lua / .js / .moon cart extension to save it in text format [PRO version]
run - Run current project
resume - Resume last loaded project
dir - Show list of local files
cd - Change directory
mkdir - Make directory
folder - Open working directory in OS (Windows, Linux, MacOSX)
add - Show file open dialog to add file to TIC
del <file> - Delete from the filesystem
get <file> - Show file save dialog to download file
export [html | native | sprites | cover | map] - Export cart to HTML or as a native build, or export sprites or cover as .gif images
import [sprites | cover | map] - Import sprites or cover from .gif
cls - Clear screen
demo - Install demo carts
version - Show the current version (0.60.3)
config [save | default] - Show config.tic file editor, use save param to save current configuration, use default to edit default cart template
surf - Open carts browser
menu - Show game menu where you can setup keyboard/gamepad buttons mapping (0.80)For those operating systems that support it, tab completion and command history is available in the console.
...
Special Functions
TIC - Main function called at 60 fps
SCN - Called on every line render, has one parameter
OVR - Called on every frame. You can draw things like a HUD on a separate layer (with own palette)Functions
print - Print string with system font
font - Print string with font defined in foreground sprites
clip - Set screen clipping region
cls - Clear the screen
pix - Set/Get pixel color on the screen
line - Draw line
rect - Draw filled rectangle
rectb - Draw rectangle border
circ - Draw filled circle
circb - Draw circle border
spr - Draw sprite by ID, can be rotated or flipped
btn - Get gamepad button state in current frame
btnp - Get gamepad button state according to previous frame
sfx - Play SFX by ID on specific channel
key - Get keyboard button state in current frame
keyp - Get keyboard button state relative to previous frame
map - Draw map region on the screen
mget - Get map tile index
mset - Set map tile index
music - Play music track by ID
peek - Read a byte value from RAM
poke - Write a byte value to RAM
peek4 - Read a half byte value from RAM
poke4 - Write a half byte value to RAM
reset - Reset game to initial state (0.60)
memcpy - Copy bytes in RAM
memset - Set byte values in RAM
pmem - Save integer value to persistent memory
trace - Trace string to the Console
time - Returns how many milliseconds have passed since game started
tstamp - Returns the current Unix timestamp in seconds (0.80)
mouse - Get XY and press state of mouse/touch
sync - Copy modified sprites/map to the cartridge
tri - Draw filled triangle
textri - Draw triangle filled with texture
exit - Interrupt program and return to console
fget - Returns sprite flag (0.80)
fset - Set sprite flag (0.80)"---
fantasy consoles
https://github.com/paladin-t/fantasy
https://lobste.rs/s/lp4jgz/i_am_trying_collect_list_toy_processors
"similar to PICO-8, TIC-80, LIKO-12"
"Some fantasy consoles and computers that stood out to me in my brief research into them so far are the PICO-8, the TIC-80, the PQ93, the SCRIPT-8, the LIKO-12 and Pixel Vision 8 (PV8). Many of these use popular languages like C# and JavaScript?."
https://itch.io/c/195696/fantasy-consoles
" Into JavaScript?? You might be interested in Phaser. Comfortable with C++ or C#? Unity, Unreal Engine, or Godot might be a good match for you. Proficient with Python? Check out Pygame or Godot. Dangerous with Java? Take a look at libGDX. In love with Lua? Check out LÖVE?, Defold, and the wonderful world of fantasy consoles like LIKO-12, PICO-8, Pixel Vision 8, and TIC-80. "
https://www.saashub.com/compare-tic-80-vs-pico-8
"
What are some alternatives? When comparing TIC-80 and PICO-8, you can also consider the following products
Pixel Vision 8 - 8-bit fantasy console built on top of an open source C# SDK
8bitworkshop - Online IDE for Atari 2600 development based on Javatari.
Voxatron - Voxel-based fantasy console
Aseprite - Aseprite is an art program dedicated to the creation of pixel art.
LIKO-12 - Fantasy console built with the Love 2D game engine.
Pyxel - Retro game engine for Python inspired by fantasy consoles.
"
https://www.reddit.com/r/pico8/comments/e6kq9y/why_is_pico8_more_popular_then_others/
---
libuv rktio
---
this blog post recommends some date libs at the bottom: luxon day.js date-fns js-joda https://momentjs.com/docs/#/-project-status/
---
most fns in https://www.jsoftware.com/help/learning/01.htm are useful and should indeed get special syntax even if only in the form of those capital letter operators i was thinking of (should those really be operators when they are connected e.g. "fnMlist" (map fn list) (which would prevent camelCase), or only when they are alone e.g. "fn M list"?)
probably lots of useful fns in further chapters in that book, too
---
svnpenn 3 hours ago [–]
I like this idea, but I dont know if Hyper is the best package to go with. Hyper occupies part of the Rust ecosystem that I think suffers from package bloat, like much of NPM. For example, currently Hyper requires 52 packages:
autocfg, bitflags, bytes, cfg-if, fnv, fuchsia-zircon, fuchsia-zircon-sys, futures-channel, futures-core, futures-sink, futures-task, futures-util, h2, hashbrown, http, http-body, httparse, httpdate, indexmap, iovec, itoa, kernel32-sys, lazy_static, libc, log, memchr, mio, miow, net2, pin-project, pin-project-internal, pin-project-lite, pin-utils, proc-macro2, quote, redox_syscall, slab, socket2, syn, tokio, tokio-util, tower-service, tracing, tracing-core, try-lock, unicode-xid, want, winapi, winapi-build, winapi-i686-pc-windows-gnu, winapi-x86_64-pc-windows-gnu, ws2_32-sys
reply
nicoburns 2 hours ago [–]
Part of this is just crates being broken up more in Rust. For example the `http` crate only contains trait (interface) definitions. They break down like so:
Platform integration: libc, winapi, winapi-build, winapi-i686-pc-windows-gnu, winapi-x86_64-pc-windows-gnu, ws2_32-sys, fuchsia-zircon, fuchsia-zircon-sys, kernel32-sys, redox_syscall
Primitive algorithms: itoa, memchr, unicode-xid
Proc macro / pinning utilities: proc-macro2, autocfg, cfg-if, lazy_static, quote, syn, pin-project, pin-project-internal, pin-project-lite, pin-utils
Data structures: bitflags, bytes, fnv, hashbrown, indexmap, slab
Core Rust asyncio crates: mio, miow, iovec, tokio, tokio-util, futures-channel, futures-core, futures-sink, futures-task, futures-util,
Logging: log, tracing, tracing-core
The following are effectively sub-crates of the project: http, http-body, httparse, httpdate, tower-service, h2
Not sure what these are for: net2, socket2, try-lock, want
reply
---
---
"By marking themselves as no_std, programs confine themselves to the functionality found in libcore. This functionality, in turn, makes no system assumptions — and in particular, performs no heap allocations. This is not easy for a system to do; it requires extraordinary discipline by those developing it (who must constantly differentiate between core functionality and standard functionality) and a broad empathy with the constraints of embedded software. Rust is blessed with both, and the upshot is remarkable: a safe, powerful language that can operate in the highly constrained environment of a microcontroller — with binaries every bit as small as those generated by C. " -- [1]
libcore: https://doc.rust-lang.org/core/
---
rust pretty print in hex:
" Two years ago, I mentioned that I love format!, and in particular the {:?} format specifier. What took me longer to discover was {:#?}, which formats a structure but also pretty-prints it (i.e., with newlines and indentation). This can be coupled with {:#x} to yield {:#x?} which pretty-prints a structure in hex. " -- [2]
---
"Rust refuses to paper over the cracks in computing’s foundation for sake of expediency. If this can feel unnecessarily pedantic (can’t I just have a timestamp?!), it is in multi-platform support where this shines: software that I wrote just… worked on Windows. (And where it didn’t, it was despite Rust’s best efforts: when a standard library gives you first-class support to abstract the path separator, you have no one to blame but yourself if you hard-code your own!)" -- [3]
---
"
Concurrency
It shouldn't come as a surprise that a lot of effort in the new C specification went into making it easier to write multithreaded code in C. The biggest change is that C now has a memory model that accepts the fact that variables may be modified from multiple threads. This comes with a set of explicit memory orderings and barriers, which I discussed in my earlier article "Understanding C11 and C++11 Atomics."
Interestingly, C11 comes with a set of standard functions for creating threads, as well as their basic synchronization primitives, which has been the cause of a lot of grumbling. The interfaces are semantically more or less equivalent to the POSIX threading APIs, but with some subtle differences.
C11 also includes some poor design choices, such as specifying timeouts in absolute times. This leads to some interesting problems when; for example, when the Network Time Protocol changes your clock time backward. In general, expiring from a timeout too early is mildly irritating, whereas a timeout that's never reached can cause code to lock up completely. With absolute timeouts, the C11 versions opt for the worse option.
The rationale for C11 threading support is twofold. First, since C is now designed to support concurrency, it would be a bit silly if the standard didn't provide any mechanism for actually spawning multiple threads. Second, the POSIX thread API isn't universal. The standards committee hopes that the C11 threading APIs will be implemented for embedded environments on non-POSIX systems. Other systems are expected to implement them on top of their native threading APIs.
The other thread-related change is the inclusion of the _Thread_local storage qualifier, which means that each thread gets a private copy of a variable. Unfortunately, as with the GCC version of this feature, it's incredibly difficult to use correctly. The POSIX threading APIs for thread-local storage allow you to register a cleanup function to avoid leaks when a thread exits. _Thread_local variables lack this capability; so, for example, they can't safely contain pointers to objects that need to be cleaned up when the thread exits. " -- [4]
---
"
This implements an approximation to C++ vector<> for C, in that it provides a generic definition for dynamic arrays which you can still access in a typesafe way using arr[i] or *(arr+i). However, it is simply a convenience wrapper around the common idiom of of keeping a set of variables (in a struct or globals) which store - pointer to array - the length of the "in-use" part of the array - the current size of the allocated array I find it to be the single most useful non-built-in-structure when programming in C (hash tables a close second), but to be clear it lacks many of the capabilities of C++ vector<>: there is no range checking, the object address isn't stable (see next section for details), the set of methods available is small (although the file stb.h has another implementation of stretchy buffers called 'stb_arr' which provides more methods, e.g. for insertion and deletion). " -- [5]
---
gui lib
https://eugenkiss.github.io/7guis/tasks/
---
https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/symbol-and-operator-reference/
---
https://github.com/darklang/tablecloth
" Standard library#
Most of the code in Dark uses Tablecloth, which has the same interface for Bucklescript and native OCaml.
A lot of the backend uses Core, one of the most popular standard libraries for OCaml. The Jane Street Core library has three flavors: Base, Core_kernel and Core, each with progressively more expansive functionality. The native version of Tablecloth is built on top of "Base". The Dark backend typically uses Core_kernel as we have not transitioned to Tablecloth fully.
Note: we try to use Core_kernel directly when implementing the language and standard libraries, as Tablecloth is still in flux and has not yet reached stability. "
---
[6] makes the point that we need the efficiency of clojure transducers with the syntax of ->>
e suggests a syntax of something like
(t-> payments (filter cash?) (map :amount) (transduce +))
and provides a macro to give that (which is the example macro i gave in the previous section)
---
Go's new file system lib:
https://tip.golang.org/pkg/io/fs/
https://go.googlesource.com/proposal/+/master/design/draft-iofs.md
---
high-speed mutation friendly SortedSet?
https://blog.discord.com/using-rust-to-scale-elixir-for-11-million-concurrent-users-c6f19fc029d3
kilotaras 19 hours ago [–]
Articles SortedSet?[0] is basically a pseudo-BTree with fixed depth = 2 and fixed max_size of leafs. This gives O(log n) search and O(sqrt n) inserts [1].
It would mean that insertion at beginning is worst case scenario (split + need to move all buckets), but timing of insert is actually dominated by adding size of the buckets to calculate final index.
Spending a little bit of time on research, finding https://en.wikipedia.org/wiki/Order_statistic_tree and just using G++ implementation would probably yield better result and less code to support.
G++ has __gnu_pbds which add O(log n) "find_by_order" and "order_of_key" to trees, e.g. [2].
[0] https://github.com/discord/sorted_set_nif/blob/master/native...
[1] Technically O(n/max_size + max_size) but we can assume that max_size is selected to be ~sqrt n
[2] https://www.ideone.com/8mzxGR
reply
---
kornel 1 month ago
| link |
WASM is sandboxed to maximally hermetic paranoid level. It doesn’t support any communication with the outside world, except callbacks exposed by the interpreter. You can’t write WASM for your operating system. There is an emerging abstraction layer called WASI, which is like a very basic tiny abstract operating system.
In practice that means you can’t use use your language’s standard library, not even printf, unless it’s been rewritten for the WASI “operating system”.
---
https://github.com/louthy/language-ext
---
" Very few applications need random read/write access to files. Most of the time, you need to read an entire file in, or write an entire file out via the streaming access APIs. This core fact of typical usage is why I think so many application developers have naive expectations about filesystem behavior.
Read-only random-access is well served by mmap() and pread(). " [7]
---
"I write database engines for Linux and the filesystem situation really is a train wreck. ... Ironically, I now tend to borrow the low-level storage interface from database kernels, which abstracts the filesystem mess, for all code that needs to work with storage even if it isn't a database. It provides a saner interface with more consistent guarantees and often better performance. "
ChrisSD? on July 23, 2019 [–]
I would only add that mmap should be used with care[0] and pread should be preferred.
[0]: https://www.sublimetext.com/blog/articles/use-mmap-with-care
jcranmer on July 23, 2019 [–]
It should be noted that much of the problems with mmap cited there aren't actually mmap's fault, but the fact that I/O errors become POSIX signals, and the API for POSIX signals really blows. (POSIX signals are probably even more ripe than filesystems for needing a different approach).
I'm surprised that page didn't mention the other issue with mmap, which is concurrent file access.
wahern on July 23, 2019 [–]
How would you signal errors accessing virtual memory?
In the context of a VM error, the only deficiency with POSIX signals I can think of off-hand is that POSIX only permits global signal handlers--not per thread--but that's only an issue for the multithreaded case.
jcranmer on July 24, 2019 [–]
I'm partial to something more akin to SEH for the synchronous signals (SIGILL, SIGFPE, SIGSEGV, SIGBUS, SIGTRAP). Basically, define that synchronous signals are handled per-thread in a manner similar to try/catch, although you need an extra catch type that amounts to "retry the operation" in addition to "rethrow" and "swallow the exception".
wahern on July 24, 2019 [–]
Can you restart execution at the point of the fault using SEH? A brief skim of the documentation doesn't seem to suggest it's possible as a general matter. If I wanted to implement a dynamically growable stack structure in a way that let me resume at the point of the fault, preserving program state, how would I do that?
I ask because I think sometimes people conflate POSIX signals offering poor semantics (i.e. reentrancy issues) with POSIX signals being too low-level. I can imagine how I might implement SEH using POSIX signals (though a per-thread handler would be really nice), but not vice-versa (though maybe it is possible).
As I understand it, there's a long history behind signals relating to interrupt vs polling software system models. Signals as they exist in Unix were a very early implementation of the interrupt driven model in the context of a kernel<->user space interface. But Unix's process and I/O concepts were perhaps too convenient so the value-add of the signals model was minimal; Unix ended up evolving in directions that didn't require the interrupt abstraction (at least, not until decades later). This history explains why POSIX signals are so low-level and the lack of comprehensive runtime treatment.
Note that they're only low-level by today's standards. At the time they were very high-level--a signal interrupt magically preserved your process state (stack and program counter) and would magically resume the process when returning from the handler, which could be a simple C function. Even better, this could occur recursively! And it's worth mentioning that all kernel interfaces were and remain async-signal safe (e.g. dup2 is atomic even from the perspective of an async signal).[1] The lack of consistent and convenient treatment by the runtime comes from the fact that much of the runtime we're most familiar with came later; threads came way later. When they came about people had already moved away from signals, perhaps because they saw that it was too much work to make the interrupt driven model work well at a high-level.
[1] In classic Unix style it did all this with the most minimal of kernel and user space code, pushing process state onto the user space stack and relying on an in-process trampoline to restore program state (which is how recursion could be supported without any complexity in kernel space).
skybrian on July 23, 2019 [–]
I'm curious, what low-level storage interfaces do you recommend?
jandrewrogers on July 23, 2019 [–]
I use the same one I use in the database engines I work on, so not open source. It provides a flexible storage abstraction on top of Linux with a lot of control over performance and behavior. It is a bit like an explicit mmap() implementation. Underneath the hood it uses direct (i.e. no kernel caching) pread/pwrite.
---
quickquestion42 on July 23, 2019
| parent | favorite | on: Files Are Fraught with Peril |
Where can I learn more about the rename() trick not being safe?
...
tfha on July 23, 2019
| parent | favorite | on: Files Are Fraught with Peril |
We wrote a program where atomicity was mission critical, and tested the hell out of power loss recovery. I can confirm experimentally that the rename trick is not good enough.
It was nearly good enough on Linux. Corruption was very rare, but we still caught issues. On Windows corruption happened something like 1 in 10 poweroffs.
For low performance ACID we write the whole file out twice and fsync between each write. First write is to the backup file, second is to the primary file. This method passed our tests.
For high performance, we use another strategy entirely that's a lot more involved.
---
" According to the article, open+write+fsync+close+dir-fsync is indeed safe on ext4 on any new-ish Linux kernel (on older kernels, fsync() wouldn't report errors in some cases, e.g. if it happened after the FS attempted and failed to write the data before the fsync).
Not sure what it has to do with rename though. "
https://news.ycombinator.com/item?id=25094828
---
seanwilson on Nov 15, 2015 [–]
I wish a library like underscore or lodash was built into JavaScript? to be honest as JavaScript? lacks built-in functions for common tasks. Any time a library boasts "no dependencies!", they end up doing their own implementations of a bunch of lodash functions in a more verbose and less safe manner. The functions in lodash are general enough and quick enough to understand if you've not seen them before that it's much better to use them than writing your own in my opinion.
---
https://github.com/redplanetlabs/specter
---
"Basically computers that start looking and acting more like clusters. Smarter memory and caching, zero-copy/fast-copy-on-mutate IPC as first-class citizens. More system primitives like io_uring to facilitate i/o."
---
https://news.ycombinator.com/item?id=25121705
" We build all our desktop products using Swing, and it works pretty well ... I have a long history with Swing and as painful as it is to use sometimes (e.g. "There are Swing developers with low pain tolerances?") it's astonishing how productive Swing feels compared to React even so many years later. ... Regarding JavaFX?... Honestly, I really like it. I have a personal project that was originally in Swing and I ported it to JavaFX? because HiDPI? is in a lot better shape there (though I think Swing has since figured it out?). And it's a lot better in many ways. I love-love-love using SceneBuilder? for designing the view layer and the Observable pattern, I think, yields a lot of nice QOL/verbosity-reducing things.
That said, I think it did have a slightly higher O() for frustration than Swing's O(1). And I think the community, as a whole, just understands it less (less prior art etc).
So if someone said "build a usable high-quality GUI asap or you die" to me, I'd go to Swing immediately.
manishsharan 2 hours ago [–]
Ahh ! Gridbaglayout! I hated it so much ! Then I had to work with CSS and JS and I began to realize the genius of the Gridbaglayout!
reply
monknomo 17 minutes ago [–]
I am right there with you on gridbaglayout. And tools like WindowBuilder? make it so fast to generate basically acceptable guis
reply
specialist 3 hours ago [–]
My buddy worked on a niche volumetric (voxel) renderer for ages. I helped with some of the OpenGL? bindings and plumbing.
Java Web Start (JNLP) worked great. It just worked. His customers loved him.
I've always wondered why something like desktop JNLP didn't happen.
reply
gorjusborg 2 hours ago [–]
I owned development of a JNLP app around a decade ago.
JNLP did a good job of solving the app distribution/update problem, but required that you already had the JRE installed. Not too much different from assuming a browser is installed, but I found it much less likely that a user would take the initiative to install a JRE than a browser. They just don't understand what they are doing or why in the case of the language runtime.
That said, it did work very well, in my experience.
reply
monknomo 13 minutes ago [–]
I'll echo this. I owned a jnlp app, and most of the support we had was around walking people through installing a sufficiently correct jre. Not always easy to do that
reply
reply
specialist 4 hours ago [–]
Swing is mostly fine. Ideologically, it's cousins with Qt.
I don't think Swing's MVC strategy worked out. It was worth trying. But maybe 40%-50% of the framework could be safely removed.
Some kind of DSL would be nice. JavaFX? ain't it. In my own noodling, Swing's UI component hierarchy needed to be refactored to be fully composable. By the time I created enough shims to make Swing truly composable, I wish I'd started from scratch. (I haven't done Swing work in ages, so maybe this happened.)
An event loop is the biggest thing missing from Swing and other UI toolkits of that era. Meaning all modifications are handled by the framework, no client code (or non event loop threads) can modify the UI. My bestie George Smith's Juipeter (?) took the Win32 API approach, preserving the Swing API and all dispatch was handled under the hood. This is kinda how a browser's event loop is implemented. I much prefer an exposed event loop and Command objects. (It's easier to stepwise debug. Every desktop app I've ever created also had Undo/Redo; so you're gonna do Command objects one way or another.) And then make the framework pretty with some kind of DSL over the top.
...
"
swiley 5 hours ago [–]
You would have thought after 40 years there would be some standard GUI toolkit all the major OSes would adopt but stuff like swing, html, and TCL/tk are all we have...
and of course vt100 escape sequences for some reason.
reply
barnacled 4 hours ago [–]
I guess not globally adopted but Qt to me is a pretty workable cross platform solution, nicely performant and reasonably easy to work with with great community support and problem googleability as well as pretty damn nice docs.
reply
ta988 3 hours ago [–]
Unfortunately not easily usable with the JDK...
reply
CyberDildonics 1 hour ago [–]
> stuff like swing, html, and TCL/tk are all we have...
They are all you have if you don't know about FLTK, QT, Juce, GTK, wxWidgets and many others.
reply
vbezhenar 6 hours ago [–]
When I wanted to write application, I used Swing and I didn’t find it horrible. Quite the opposite. I didn’t care about the looks, though, it was not a major concern. That was before Java 8. I could imagine that writing callbacks would be even easier with lambdas.
When people don’t laugh at Electron applications, I don’t know what could be wrong with Swing. Nobody cares about native toolkits anymore (what is native toolkit for Windows or Linux?).
reply
hutzlibu 5 hours ago [–]
Yeah well, " I didn’t care about the looks, though, it was not a major concern"
There is the difference. This post is for people who do care about the looks. (or do care, that their users care) Some users don't care, which is fine to serve them, what they want, but most actually do.
blablabla123 50 minutes ago [–]
I really liked Processing [1] It's possible to create practically every thinkable visualization from scratch with surprisingly little design knowledge/programming effort. Not only is it far more intuitive and flexible than d3.js but also it can be directly called as Java program. Apparently the Arduino UI is a fork of its editor. FWIW it's hard to tell that the Arduino IDE is not a native app.
[1] https://en.wikipedia.org/wiki/Processing_(programming_langua...
reply
suyash 14 minutes ago [–]
Processing is getting an update Processing 4 and it would be nice if they incorporate this library.
reply
didibus 29 minutes ago [–]
I've found apps made with JUCE https://juce.com/ always look great and have really good responsiveness.
Now on the JVM front, I think one problem it has for desktop apps is the memory usage of the JVM is just terrible for desktop apps. It's clearly optimized for backend services assuming they are the only app running trying to squeeze all performance out. But a desktop app needs to be lean, and currently I'm not sure the JVM is tuned to those use cases, and Java is missing a few features for lightweight objects that could help as well with memory usage.
reply
johnnycerberus 2 hours ago [–]
I think you misinterpreted what the author aims to do. He states in the beginning that the main reason for why we don't have a good UI toolkit on the JVM is because every existing solution is bringing its own drawing library and with it their very own limitations. He advocates for a drawing library that will be supported community-wide, something like Spring is.
React Hooks, SwiftUI? and Flutter are modern choices that popularized the declarative approach and the author expresses its frustration that until now there was no interest in developing one for the JVM, which has all the pluses that it needs to create an ecosystem around one. Java, Kotlin, Scala and Clojure will all benefit from it. I think he is right, there is no modern graphics API on the JVM right now to do an UI toolkit or 2D/3D visualisations.
---
https://en.wikipedia.org/wiki/SipHash
---
"I’m also liking what SerenityOS? is doing with the LibCore?/LibGfx?/LibGui? stuff. "
---
"
friendlysock 11 hours ago
| link | flag |
Digging into IPC a bit, I feel like Windows actually had some good stuff to say on the matter. https://docs.microsoft.com/en-us/windows/win32/ipc/interprocess-communications https://docs.microsoft.com/en-us/windows/win32/winmsg/about-messages-and-message-queues
I think the design space looks something like:
Messages vs streams (here is a cat picture vs here is a continuing generated sequence of cat pictures)
Broadcast messages vs narrowcast messages (notify another app vs notify all apps)
Known format vs unknown pile of bytes (the blob i’m giving you is an image/png versus lol i dunno here’s the size of the bytes and the blob, good luck!)
Cancellable/TTL vs not (if this message is not handled by this time, don’t deliver it)
Small messages versus big messages (here is a thumbnail of a cat versus the digitized CAT scan of a cat)I’m sure there are other axes, but that’s maybe a starting point. Also, POSIX signals. Not in my OS.
"
" zie edited 11 hours ago
| link | flag |
Sane atomic file I/O, guaranteed by the OS to just work. This: https://danluu.com/deconstruct-files/ should not have to exist.
13 "
"
13 zge edited 11 hours ago
| link | flag |
I like to think about these things, but don’t have much hope. Here are my points:
Networking shouldn’t be an afterthought. Distributed computing should not be as difficult as it is. Transparently interacting or using resources from other systems should be something you don’t have to think about. I don’t care about hardware. I don’t care about CPU architectures. I don’t care about GPUs. I don’t care about drivers. All computers form an transnational turing machine.
[https://en.wikipedia.org/wiki/Object-capability_model Object capabilities] should be a primitive concept. Imagine sharing a screen: That shouldn’t be the hassle it is, you should just be able to give someone read access to a segment or the whole display. The same applies to Files (but we probably shouldn’t have files), Hardware access, etc.
Hypertext should be a everywhere. The web has shown how attractive the idea is, but browsers are cursed to contain it, which is getting harder and harder. Project Xandu had good ideas about this, and HTTP is a shallow copy. We need complex links that can point to miscelanious parts of the system, and ideally also have back-references. You probably want a lof of cryptography for something like thise, to avoid the centralisation of power....In some sense, I like to think of it like Plan 9, without the Unix legacy, but that seems to simplistic. The interesting thing about Unix, is that despite it’s limitations, it creates the fantasy of something better. Inbetween it’s ideal power and it’s practical shortcomings, one can imagine what could have been. ...
~ spc476 5 hours ago
| link | flag |
Networking: QNX was network transparent. It was wild running a command on computer 1, referencing a file from computer 2, piping the output to a program on computer 3 which sent the output to a device on computer 4. All from the command line. The IPC was fast [1] and network transparent, and used for just about everything.
Hypertext: The only operating system I know of that uses extensive form of hypertext is TempleOS (I don’t think it’s HTML but it is a form of hypertext) that extends pervasively throughout the system.
Logic and UI: There are bits and pieces of this in existence. AmigaOS has Rexx, which allows one to script GUI programs. Apple has (had?) something similar. Given that most GUI based programs are based around an event loop, it should be possible to pump events to get programs to do stuff.
Programming: True, but there is Excel, which is a programming language that doesn’t feel like one. Given an easy way to automate a GUI (similar to expect on the command line), and teaching people that computers excel (heh) at repeated actions could go a long way in giving non-programmers power.[1] In the early-to-mid 90s, I had friends that worked at a local software company that wrote and sold custom X Window servers. Their fastest X Window server ran on QNX.
...
bfiedler 9 hours ago
| link | flag |
Your first point is pretty much what Barrelfish is designed for, go check it out!
---
" To tackle the API issue I have designed Glommio (formerly known as Scipio), a Direct I/O-oriented thread-per-core Rust library. Glommio builds upon io_uring and supports many of its advanced features like registered buffers and poll-based (no interrupts) completions to make Direct I/O shine. For the sake of familiarity Glommio does support buffered files backed by the Linux page cache in a way that resembles the standard Rust APIs (which we will use in this comparison), but it is oriented towards bringing Direct I/O to the limelight. " -- https://itnext.io/modern-storage-is-plenty-fast-it-is-the-apis-that-are-bad-6a68319fbc1a
---
explains the io_uring interface and why it's good:
https://www.scylladb.com/2020/05/05/how-io_uring-and-ebpf-will-revolutionize-programming-in-linux/
some excerpts:
" Here are some of the operations that io_uring supports: read, write, send, recv, accept, openat, stat, and even way more specialized ones like fallocate. ... It is flexible and extensible: new opcodes are being added at a rate that leads us to believe that indeed soon it will grow to re-implement every single Linux system call. ... By design, the interfaces are designed to be truly asynchronous. With the right set of flags, it will never initiate any work in the system call context itself and will just queue work. This guarantees that the application will never block. It works with any kind of I/O: it doesn’t matter if they are cached files, direct-access files, or even blocking sockets. That is right: because of its async-by-design nature, there is no need for poll+read/write to deal with sockets. One submits a blocking read, and once it is ready it will show up in the completion ring. ... The io_uring interface works through two main data structures: the submission queue entry (sqe) and the completion queue entry (cqe). Instances of those structures live in a shared memory single-producer-single-consumer ring buffer between the kernel and the application. The application asynchronously adds sqes to the queue (potentially many) and then tells the kernel that there is work to do. The kernel does its thing, and when work is ready it posts the results in the cqe ring. ... The application, whenever it wants to check whether work is ready or not, just looks at the cqe ring buffer and consumes entries if they are ready. There is no need to go to the kernel to consume those entries. "
note: there are two queues; the submission queue has the user adding to it and the system removing from it, and the completion queue has the system adding to it and the user removing from it:
---
adzm 10 hours ago [–]
This feels very very similar to IO completion ports / iocp on Windows. More modern versions of Windows even has registered buffers for completion which can be even more performant in certain scenarios. I'm looking forward to trying this out on Linux.
I'm curious to see how this might work its way into libuv and c++ ASIO libraries, too.
reply
ithkuil 9 hours ago [–]
io_uring allows the kernel and the user program to communicate purely via shared memory without having to perform a system call, i.e. a context switch to the kernel.
Do windows completion ports also work that way or do they involve a system call to be performed in order to consume completion events?
reply
Matthias247 4 hours ago [–]
Windows registered IO (RIO) does imho the same (https://docs.microsoft.com/en-us/previous-versions/windows/i...). When enqueuing reads/writes with RIO there at least exist flags to specify that the kernel should not immediately be woken up, and thereby to batch syscalls as with io_uring.
reply
ta8645 9 hours ago [–]
You do still need a single system call (io_uring_submit) to submit each batch of entries in the submission queue.
Edit: actually no it's not required in all cases. Thanks for the correction.
reply
ithkuil 9 hours ago [–]
I only read about io_uring without yet having a chance of actually using it so take this with a grain of salt:
I read that io_uring has two modes, one where you signal via a system call and another that uses memory mapped polling.
https://unixism.net/loti/tutorial/sq_poll.html states:
> Reducing the number of system calls is a major aim for io_uring. To this end, io_uring lets you submit I/O requests without you having to make a single system call. This is done via a special submission queue polling feature that io_uring supports.
reply
yxhuvud 3 hours ago [–]
Submit is not a syscall. io_uring_enter is the only syscall that is used while running a ring. That one may submit, wait or both at the same time. Strictly speaking it isn't necessary but to avoid it you require elevated privileges.
reply
WJW 9 hours ago [–]
You can ask the kernel to poll the submission queue and skip io_uring_submit too. (Though you need elevated privileges to do this IIRC)
reply
leetrout 9 hours ago [–]
libuv is tracking in https://github.com/libuv/libuv/issues/1947
reply
cycloptic 2 hours ago [–]
Tracking that issue was a motivator for me to begin adding support to glib: https://gitlab.gnome.org/GNOME/glib/-/issues/2084
reply
cyphar 6 hours ago [–]
You're quite right -- it's basically the same idea as IOCP on Windows, kqueue on FreeBSD?, and Event Ports on Solaris.
reply
Matthias247 4 hours ago [–]
Isn't kqueue for sockets still readiness based? I know most runtimes (like libuv) just use it in the same fashion as epoll, and await readability/writeability through the queue. Not sure if it also has completion based options.
reply
cyphar 2 hours ago [–]
You're quite right, I muddled things up (and now the edit window has elapsed). epoll is the Linux equivalent of kqueue, IOCP, and Event Ports (all readiness based). Not sure how I screwed that one up...
reply
ncmncm 1 hour ago [–]
It is already integrated with asio. Third-party, of course, because that's the whole point: io_uring does not need to know anything about asio, nor does asio need to know anything about io_uring, to get optimal performance.
It's all on github, with accompanying CppCon? talk. Asio, by the way, will be C++23's network layer.
reply
---
https://emscripten.org/docs/porting/guidelines/api_limitations.html
says
"Emscripten supports libc networking functions but you must limit yourself to asynchronous (non-blocking) operations. This is required because the underlying JavaScript? networking functions are asynchronous."
so i guess sometimes aio is provided before blocking operations? Seems odd to me b/c if you have aio you can just sleep until the operation completes, unless there is no way for you to tell when it completes?
mb see
https://emscripten.org/docs/porting/files/file_systems_overview.html https://emscripten.org/docs/api_reference/Filesystem-API.html#filesystem-api-filesystems https://emscripten.org/docs/api_reference/emscripten.h.html#emscripten-h-asynchronous-file-system-api
https://emscripten.org/docs/porting/files/file_systems_overview.html says "If you need to fetch other files from the network to the file system then use emscripten_wget() and the other methods in the Asynchronous File System API. These methods are asynchronous and the application must wait until the registered callback completes before trying to read them. "
oh i see, they are saying you can't poll for completeness, you can only register a callback
---
https://please.build/lexicon.html#python-builtins
---
" I’m also liking what SerenityOS? is doing with the LibCore?/LibGfx?/LibGui? stuff. A “standard” set of stuff seems really cool because you know it will work as long as you’re on SerenityOS?. While I’m all for freedom of choice having a default set of stuff is nice.
"
---
"Newton soups: no need for a filesystem when you have a system-wide graph database."
---
"Expand on the idea behind Erlang and processes having a common messaging behavior. Almost everythin becomes “gen_server” like. I know Plan 9 has this idea of everything as a file server, but abstract that a little bit and create the same interface across the board."
c-cube 4 days ago
| link | flag |
If there’s a thing to take from Erlang, I think it’d be to use structured data (tuples, atoms, ints, strings/bitstrings) in messages, not just a stream of bytes like unix does. I think an OS where programs communicate via Erlang’s data structures would be much better and richer than anything built on unstructured bytes.
---
"Kill the VT100. Command lines shouldn’t be limited by curses, but display rich objects too. This complements what people are talking about with structured IPC. Emacs shows ideas of how this can work, albeit in a crude manner.
"
---
---
https://doc.rust-lang.org/core/
---
https://github.com/StephenCleary/AsyncEx
---
