Bayle Shanks's website: proj-oot-ootLibrariesNotes8

" 1. Learn Key Syscalls

You should know what the following 12 key system calls do, which you'll see regularly in strace output. Test your knowledge! How many of these do you know? Mouse-over for answers. syscall what it does read write open close fork exec connect accept stat ioctl mmap brk

Click here to reveal all. Each syscall has a man page, so if you are at the command line, it should only take a few seconds to jog your memory.

There are variants of these syscalls, so you may see "execve" for "exec", and "pread" as well as "read". There should be man pages for these, too.

---

" Category of Syscall Syscalls Seen in Critical Sections Frequency in Critsecs: MySQL? Firefox Time gettimeofday , clock gettime 3.91% 70.18%

Filesystem read* , write* , open , close , lseek , access , dup , mkdir , ftruncate , fsync , writev , pread* , pwrite* , stat , fstat , fcntl , getdents , getcwd , fdatasync , mmap* , munmap* , mprotect* 53.79% 28.75%

Process Memory brk , mmap* , munmap* , mprotect* 31.03% 0.32%

Process Maintenance waitpid , clone , sched setscheduler , sched get priority max , sched get priority min , rt sigaction , rt sigprocmask , tgkill 8.97% 0.32%

Communication ioctl , socket , pipe , read* , write* , pread* , pwrite* 2.07% 0.40%

System Info sysinfo , uname 0.23% 0.03%

Table 1. Syscalls Seen in Critical Sections Six different categories of syscall were seen in critical sections in MySQL? and Firefox; their dynamic distributions in the critical sections of each workload are shown at right. Communications syscalls, which are most difficult to make transaction-safe, are uncommon. Syscalls marked with asterisks belong to several categories "

---

" vectors. The process of transforming filtered system calls into the histogram representation is depicted in Python algorithm 5.1.

Histogram Raw Normalised recvfrom 25284 0.2968720 write 12396 0.1455476 epoll_pwait 11581 0.1359783 futex 9995 0.1173562 getuid 6209 0.0729029 ioctl 5244 0.0615724 sendto 4512 0.0529776 read 4108 0.0482340 mprotect 1509 0.0177179 pread64 691 0.0081133 mmap 613 0.0071975 newfstatat 418 0.0049079 munmap 392 0.0046026 fstat 366 0.0042973 openat 237 0.0027827

Table 5.2: Histogram; excerpt of 15 first lines. Constructed from a 100 second benign sample of BBC News ( "

---

" Priority value Syscalls

0 socket , bind , getsockname , getsockopt , setsockopt

1 poll , select , getpeername

2 accept , recv , recvfrom , recvmsg , read

3 connect , send , sendto , sendmsg , write , writev , sendfile

4 close , shutdown , listen

Table 1: Simulation priority of common syscalls: the lower the priority value, the higher the priority. "

---

" TABLE IV. OVERVIEW OF ACTIONS PERFORMED BY CUSTOM SHARED LIBRARIES IN NATIVE CODE .

Writing log messages Performing memory management system calls, such as mmap and mprotect Reading files in the application directory Calling JNI functions Performing general multiprocess and multithread related system calls, such as fork , clone , setpriority , and futex Reading common files, such as system libraries, font files, and “/dev/random” Performing other operations on files or file descriptors, such as lseek , dup , and readlink Performing operations to read information about the system, such as uname , getrlimit , and reading special files (e.g., “/proc/cpuinfo” and “/sys/devices/system/cpu/possible”) Performing system calls to read information about the process or the user, such as getuid32 , getppid , and gettid Performing system calls related to signal handling Performing cacheflush or set_tls system calls or per- forming nanosleep system call Reading files under “/proc/self/” or “/proc/ < PID > /”, where PID is the process’ pid Creating directories "

" TABLE V. T OP FIVE MOST COMMON ACTIONS PERFORMED BY APPS IN NATIVE CODE , THROUGH SHARED LIBRARIES (SL) AND CUSTOM BINARIES (CB). F OR THE INTERESTED READER , WE REPORT THE FULL VERSION OF THIS TABLE IN [1]. SL CB Description 3,261 72 ioctl system call 1,929 39 Write file in the app’s directory 1,814 35 Operations on sockets 1,594 5 Create network socket 1,242 144 Terminate process or thread group "

" TABLE IX. G ROUPS OF JNI CALLS USED FROM NATIVE CODE . Apps Description 94,543 Get class or method identifier and class reference 71,470 Get or destroy JavaVM?, and Get JNIEnv 53,219 Manipulation of String objects 49,321 Register native method 45,773 Manipulate object reference 41,892 Thread manipulation 35,231 Call Java method 19,372 Manipulate arrays 18,601 Manipulate exceptions 14,330 Create object instance 6,918 Modify field of an object 2,203 Manipulate direct buffers 47 Memory allocation 37 Enter or exit monitor

TABLE X. T OP 10 GROUPS OF J AVA METHODS FROM THE A NDROID FRAMEWORK CALLED FROM NATIVE CODE . Apps Description 7,423 Get path to the Android package associated with the context of the caller 6,896 Get class name 5,499 Manipulate data structures 4,082 Methods related to cryptography 3,817 Manipulate native types 3,769 Read system information 3,018 Audio related methods 2,070 Read app information 1,192 String manipulation and encoding 575 Input/output related methods 483 Reflection "

" TABLE XII. T OP 10 MOST USED STANDARD LIBRARIES . Apps Name Description 24,942 libjnigraphics.so Manipulate Java bitmap objects 2,646 libOpenSLES.so Audio input and output 2,645 libwilhelm.so Multimedia output and audio input 349 libpixelflinger.so Graphics rendering 347 libGLES android.so Graphics rendering 183 libGLESv1 enc.so Encoder for GLES 1.1 commands 183 gralloc.goldfish.so Memory allocation for graphics 182 libOpenglSystemCommon.so Common functions used by OpenGL? 182 libGLESv2 enc.so Encoder for GLES 2.0 commands 181 lib renderControl enc.so Encoder for rendering control commands

TABLE XIII. T OP 10 MOST USED CUSTOM LIBRARIES . Apps Name Description 19,158 libopenal.so Rendering audio 17,343 libCore.so Used by Adobe AIR 16,450 libmain.so Common name 13,556 libstlport shared.so C++ standard libraries 11,486 libcorona.so Part of the Corona SDK, a devel- opment platform for mobile apps 11,480 libalmixer.so Audio API of the Corona SDK 11,458 libmpg123.so Audio library 11,090 libmono.so Mono library, used to run .NET on Android 10,857 liblua.so Lua interpreter 10,408 libjnlua5.1.so Lua interpreter "

" TABLE XIV. A LLOWED SYSTEM CALLS AUTOMATICALLY GENERATED USING A THRESHOLD OF 99% APPS UNAFFECTED BY THE POLICY . accept(*,*,*) access( < SYS-PATH > , F OK) access( < SYS-PATH > ,R OK) access( < SYS-PATH > , W OK) access( < SYS-PATH > ,X OK) access( < USER-PATH > , F OK) access( < USER-PATH > ,R OK) access( < USER-PATH > , R OK j W OK j X OK) bind BINDER(android.os.IServiceManager?. CHECK SERVICE TRANSACTION) brk cacheflush(*,*,0,*,*) cacheflush(*,*,0,0,*) chdir chmod( < USER-PATH > ,*) clone(child stack=*,flags=CLONE VM j CLONE FS j CLONE FILES j CLONE SIGHAND j CLONE THREAD j CLONE SYSVSEM) connect(*, f sa family=AF UNIX, path=@”jdwp-control” g ,*) connect(*, f sa family=AF INET,*,* g ,*) connect(*, f sa family=AF UNIX, path=@”android:debuggerd” g ,*) connect(*, f sa family=AF UNIX, path= < SYS-PATH > g ,*) dup dup2 epoll create(*) epoll ctl(*,*,*,*) epoll wait execve exit( < NEG INT > ) exit(0) exit group( < POS INT > ) exit group(0) fcntl64( < NON STD FD > ,F DUPFD,*) fcntl64( < NON STD FD > ,F GETFD) fcntl64(*,F GETFL) fcntl64( < NON STD FD > ,F SETFD,*) fcntl64( < NON STD FD > ,F SETFL,*) fcntl64( < NON STD FD > ,F SETLK,*) fdatasync(*) fork fstat64 fsync(*) ftruncate(*,*) futex getcwd getegid32 geteuid32 getgid32 getpeername getpgid(0) getpid getppid getpriority(PRIO PROCESS,*) getrlimit(RLIMIT DATA,*) getrlimit(RLIMIT NOFILE,*) getrlimit(RLIMIT STACK,*) getrusage(RUSAGE CHILDREN,*) getrusage(RUSAGE SELF,*) getsockname getsockopt(*,SOL SOCKET,SO ERROR,*,*) getsockopt(*,SOL SOCKET, SO PEERCRED,*,*) getsockopt(*,SOL SOCKET,SO RCVBUF,*,*) gettid getuid32 ioctl( < ASHMEM-DEV > ,*,*) ioctl(*,FIONBIO,*) ioctl( < LOG-DEV > ,*,*) ioctl(*,SIOCGIFADDR,*) ioctl(*,SIOCGIFBRDADDR,*) ioctl(*,SIOCGIFCONF,*) ioctl(*,SIOCGIFFLAGS,*) ioctl(*,SIOCGIFHWADDR,*) ioctl(*,SIOCGIFINDEX,*) ioctl(*,SIOCGIFNETMASK,*) ioctl( < STD IN/OUT/ERR > , SNDCTL TMR TIMEBASE or TCGETS, *) ioctl(*,SNDCTL TMR TIMEBASE or TCGETS,*) ioctl( < URANDOM-DEV > , SNDCTL TMR TIMEBASE or TCGETS,*) listen lseek(*,*,SEEK CUR) lseek(*,*,SEEK END) lseek(*,*,SEEK SET) lstat64 madvise(*,*,MADV DONTNEED) madvise(*,*,MADV NORMAL) madvise(*,*,MADV RANDOM) mkdir( < SYS-PATH > ,*) mkdir( < USER-PATH > ,*) mmap2 mprotect mremap(*,*,*,MREMAP MAYMOVE) munmap nanosleep open( < SYS-PATH > ,*,*) open( < SYS-PATH > ,*) open( < USER-PATH > ,*,*) open( < USER-PATH > ,*) pipe poll prctl(PR GET NAME,*,0,0,0) prctl(PR SET NAME,*,*,*,*) prctl(PR SET NAME,*,*,*,0) prctl(PR SET NAME,*,0,0,0) ptrace(PTRACE TRACEME,*,0,0) readlink( < USER-PATH > ,*,*) recvfrom recvmsg rename( < USER-PATH > , < USER-PATH > ) rmdir( < USER-PATH > ) rt sigprocmask(SIG BLOCK,*,*,*) rt sigprocmask(SIG SETMASK,*,*,*) rt sigreturn(*) rt sigtimedwait([QUITUSR1], NULL, NULL, 8) sched getparam sched getscheduler sched yield select sendmsg sendto setitimer(ITIMER REAL,*,*) setpriority(PRIO PROCESS,*, < POS INT > ) setpriority(PRIO PROCESS,*,0) setrlimit(RLIMIT NOFILE,*) setsockopt(*,SOL IP,*,*,*) setsockopt(*,SOL SOCKET,*,*,*) set tls(*,*,*,*,*) set tls(*,*,0,*,*) sigaction sigprocmask(SIG BLOCK,*,*) sigprocmask(SIG SETMASK,*,*) sigprocmask(SIG UNBLOCK,*,*) sigreturn sigsuspend([]) socket(PF INET,SOCK DGRAM, IPPROTO ICMP) socket(PF INET,SOCK DGRAM, IPPROTO IP) socket(PF INET,SOCK DGRAM, IPPROTO UDP) socket(PF INET,SOCK STREAM, IPPROTO IP) socket(PF INET,SOCK STREAM, IPPROTO TCP) socket(PF NETLINK,SOCK RAW, 0) socket(PF UNIX, SOCK STREAM, 0) stat64 statfs64( < SYS-PATH > ,*) statfs64( < USER-PATH > ,*) symlink( < USER-PATH > , < USER-PATH > ) tgkill(*,*,SIGTRAP) umask uname unlink( < USER-PATH > ) utimes vfork wait4 "

" TABLE XVII. L IST OF ALLOWED METHODS (J AVA METHODS CALLED FROM NATIVE CODE ) AUTOMATICALLY GENERATED FOR ALLOWING A MINIMUM OF 97%, 98% AND 99% OF APPS THAT REACHED NATIVE CODE . Allowed apps (%) Method 97 java.lang.Integer.doubleValue 97 android.content.ContextWrapper?.getPackageName 97 java.lang.String.getBytes 98 java.lang.Double.doubleValue 98 android.content.ContextWrapper?.getClassLoader 98 android.content.ContextWrapper?.getFilesDir 98 java.io.File.getPath 98 android.content.ContextWrapper?.getExternalFilesDir 98 android.view.WindowManagerImpl?.getDefaultDisplay 98 java.lang.String.toLowerCase 98 android.app.Activity.getWindowManager 98 java.util.ArrayList?.add 98 android.view.Display.getMetrics 98 android.app.Activity.getWindow 98 android.view.View.getWindowVisibleDisplayFrame 98 java.util.Calendar.getInstance 98 android.view.View.getDrawingRect 99 java.util.Calendar.get 99 android.os.Bundle.getByteArray 99 android.content.ContextWrapper?.getPackageManager 99 android.content.res.AssetManager?$AssetInputStream?.read 99 java.lang.Long.doubleValue 99 java.lang.ClassLoader?.loadClass 99 android.app.ApplicationPackageManager?.getPackageInfo 99 android.content.res.AssetManager?$AssetInputStream?.close 99 java.lang.Float.doubleValue 99 java.lang.Class.getClassLoader "

---

dazzawazza 1 day ago [-]

I've coded desktop apps (mostly editors and tools for games) for 25 years across X (Motif toolkit), Amiga, TOS, DOS, MFC, Cocoa, Win32, WxWidgets?, Fox Toolkit and Qt. Qt has by far been the best, most rewarding, most empowering experience. It's a great library.

Now Cocoa would win but I prefer C/C++ to Objective-C, it's close though.

Good work people.

hannofcart 22 hours ago [-]

Glad to see the update. Most people associate Qt with GUIs which is unfortunate.

I see that when people think of Qt, they think of WxWidgets?, Cocoa or MFC as alternatives. No, I submit to you that Qt framework is a more elegant, easier to use alternative to Boost as well.

This is not to say that QtQuick? or QtWidgets? aren't solid. However, the success of these two modules ends up occluding the others which to me are the real gems from the QtFramework?: QtCore? and QtNetwork?.

QtCore? provides a solid event loop, the most easy to use implementation of the observer pattern via its signal-slot mechanism, robust threading utilities and a bunch of other utilities that make writing apps in C++ an absolute breeze.

QtNetwork? for a series of networking utilities that are elegantly simple.

If I were to write a command line app or a database or server, I'd reach for Qt in a jiffy.

Qt is not just for GUIs!

Jasper_ 20 hours ago [-]

I really wish QtNetwork? was better. I recently went and had to replace QtNetwork? with our own event loop because it is extremely unreliable on Windows. The only mention of this is a footnote in the API docs:

http://doc.qt.io/qt-5/qabstractsocket.html#waitForBytesWritten

justinclift 17 hours ago [-]

Yeah. It also turns out that just having a Qt application running on a computer can tank network latency for all applications on the computer:

https://bugreports.qt.io/browse/QTBUG-40332

There is a workaround, but developers have to know about it first. And it's such an unexpected and weird behaviour that 99% of developers don't.

This was our fix (1 line of code), in case that helps anyone:

https://github.com/sqlitebrowser/sqlitebrowser/commit/73946400c32d1f7cfcd4672ab0ab3f563eb84f4e

nly 17 hours ago [-]

Yup, i work on software with a Qt GUI and it took us ages to figure out why our daemon (no Qt dependencies) would experience latency when the GUI was open.

For Windows users we now put this var in to global environment via our installer, and probably break other software in the process.

--- HN discussion of: Comparison of C/Posix standard library implementations for Linux (etalabs.net)

[1]