Matthias Kruk [Sat, 12 Aug 2023 05:13:52 +0000 (14:13 +0900)]
include/ipc: Add JSON Schema-based data validation
When an application receives a message from `ipc_endpoint_recv()',
it has to validate the contents of the message. It would be more
convenient if the validation could be performed by the ipc module,
so that the `ipc_endpoint_recv()' function only returns valid
messages to the caller.
This commit modifies the ipc module so that the user may set a
JSON Schema that is used by the ipc module to validate the data
of incoming messages. When a schema has been set, only messages
that passed validation will be returned by `ipc_endpoint_recv()'.
Matthias Kruk [Fri, 11 Aug 2023 08:33:06 +0000 (17:33 +0900)]
Makefile: Install JSON Schema files and validator
When installing toolbox, neither the JSON Schemas for IPC messages
nor the validator are installed. Thus, it is not possible to use the
validator to validate received messages. However, the validator is
necessary to implement incoming message validation in the ipc module.
This commit modifies the Makefile so that the JSON Schema files and
validator are installed by the `install' target.
Matthias Kruk [Fri, 11 Aug 2023 08:09:41 +0000 (17:09 +0900)]
debian: Add dependencies for JSON Schema validation script
The script for JSON Schema validation requires the python3 and
python3-jsonschema packages to be installed, but because the
toolbox debian package does not depend on them, they may not be
installed.
This commit makes the toolbox debian package depend on python3
and python3-jsonschema, making sure that the JSON Schema validator
can be executed.
Matthias Kruk [Sun, 11 Jun 2023 04:49:29 +0000 (13:49 +0900)]
include/ipc: Add function to list subscriptions of an endpoint
The IPC module does not provide a method to get the topics that an
endpoint is subscribed to, requiring the user to track the topics
themselves, if they need this information.
This commit adds the `ipc_endpoint_get_subscriptions()' method,
which returns the list of topics that an endpoint is subscribed to.
include/ipc: Correctly assign `topics' in ipc_endpoint_subscribe()
The `topics' argument of ipc_endpoint_subscribe() is not assigned
from an array, causing the function to subscribe only to the first
topic that is passed to it.
This commit fixes the assignment of the `topics' argument of
ipc_endpoint_subscribe() so that the function will subscribe an
endpoint to more than one topic.
include/ipc: Allow passing multiple topics to ipc_endpoint_subscribe()
The ipc_endpoint_subscribe() function can only be used to subscribe to
one topic at a time, making error checking in scripts that subscribe to
more than one topic needlessly complicated.
This commit modifies the `ipc_endpoint_subscribe()' function so that
the caller may pass more than one topic. If any of the topics could not
be subscribed, the function will leave the endpoint in the state it had
before the invocation (that is, undo subscriptions that were made by
the same call).
include/ipc: Add method for unsubscribing an endpoint from topics
The IPC module does not provide a function for unsubscribing an
endpoint from a pubsub topic, forcing a script that wishes to
unsubscribe from a topic to close the endpoint and open a new one.
This commit adds the `ipc_endpoint_unsubscribe()' method to the IPC
module, allowing an endpoint to be unsubscribed from pubsub topics.
Matthias Kruk [Sat, 11 Mar 2023 08:07:30 +0000 (17:07 +0900)]
utils/ipc-sshtunnel: Properly escape command passed to ssh
When ipc-sshtunnel spawns ipc-tap and ipc-inject over ssh, it does
not correctly escape the arguments passed to ssh, causing arguments
to be expanded where no expansion is desired.
This commit modifies the way ipc-sshtunnel so that it correctly
quotes and escapes arguments that are passed to ssh.
Matthias Kruk [Fri, 10 Mar 2023 11:16:40 +0000 (20:16 +0900)]
include/uipc: Fix version checking when dumping uipc messages
The function `uipc_msg_dump()' still references `_uipc_msg_get()'
even though it was removed from the module, rendering the function
unable to figure out the message version.
This commit changes the uipc module to use `ipc_msg_get_version()'
to query message versions.
Matthias Kruk [Fri, 10 Mar 2023 10:47:02 +0000 (19:47 +0900)]
include/ipc: Add catchall-topic that receives all pubsub messages
The current ipc implementation makes it impossible to tunnel all
pubsub messages between two systems because an endpoint would need
to subscribe all topics, including those that will be created after
the endpoint is opened.
This commit allows endpoints to subscribe the topic "*". Subscribers
that subscribe to this topic are called "taps" and receive messages
from all topics.
Matthias Kruk [Thu, 9 Mar 2023 11:11:38 +0000 (20:11 +0900)]
utils: Make ipc-tap and ipc-inject use the same endpoint
When ipc-sshtunnel is used to tunnel a pubsub topic in both directions,
ipc-tap will receive messages sent from the ipc-inject instance of the
same tunnel. This makes it impossible to use ipc-sshtunnel to relay a
topic in both directions.
This commit modifies ipc-tap and ipc-inject so that the caller may pass
an endpoint name that the scripts will use for IPC communication. This
further modifies ipc-sshtunnel so that it passes the same endpoint name
to ipc-tap and ipc-inject, causing the same endpoint to be used for
sending and receiving of messages. This causes messages from ipc-inject
not to be sent to ipc-tap.
Matthias Kruk [Thu, 9 Mar 2023 07:48:18 +0000 (16:48 +0900)]
utils: Allow ipc-sshtunnel and friends to be used for uipc messages
The scripts for tunneling IPC messages over SSH cannot be used with
uipc messages, even though the two modules share the same interface.
This commit modifies the scripts ipc-sshtunnel, ipc-inject, and
ipc-tap, so that they can handle uipc as well as ipc messages by
adding a command line option for specifying the ipc protocol to be
tunneled.
Matthias Kruk [Thu, 9 Mar 2023 06:11:33 +0000 (15:11 +0900)]
include/ipc: Don't send pubsub messages to oneself
When publishing a pubsub message, ipc_endpoint_publish() sends the
message to all subscribers, including the endpoint that is used to
send the message, if it is subscribed to that topic. However, an
endpoint should not receive its own messages.
This commit modifies ipc_endpoint_publish() so that it skips over
the sending endpoint if it is among the recipients.
Matthias Kruk [Tue, 14 Feb 2023 12:36:57 +0000 (21:36 +0900)]
debian/{postinst,postrm}: Add stat overrides for all IPC directories
The post-installation and post-removal scripts in the debian package
do not create or remove stat overrides for the directories in the IPC
directories, possibly leaving ownership and permissions in an
incorrect state.
This commit modifies the post-installation and post-removal scripts
so that stat overrides for IPC directories are correctly created and
removed.
Matthias Kruk [Tue, 14 Feb 2023 12:07:06 +0000 (21:07 +0900)]
Makefile: Correctly create IPC directories
The command that is supposed to create the directory structure for
IPC messaging is written using brace expansion, despite not being
a shell script. Because GNU Make does not support brace expansion,
it ends up creating an incorrect directory structure.
This commit changes the install rule of the Makefile to avoid brace
expansion.
Matthias Kruk [Mon, 9 Jan 2023 09:20:41 +0000 (18:20 +0900)]
include/uipc: Implement uipc module as extension of ipc module
The uipc module contains several functions that are identical with
their ipc-counterpart. This kind of code-duplication can be avoided
using the new interface/implements mechanism.
This commit modifies the uipc module so that it extends the interface
provided by the ipc module, reusing methods of the ipc module where
possible.
Matthias Kruk [Mon, 9 Jan 2023 09:17:10 +0000 (18:17 +0900)]
include/ipc: Add functions to ipc interface so it can be extended
The ipc module provides an interface, but it cannot meaningfully
be extended because the interface is missing functions that need to
be overridden by an extending module.
This commit makes several message-related functions public and adds
them to the interface that is provided by the ipc module. This
further adds the `ipc_get_root()' function, which replaces the
`__ipc_root' variable, allowing modules that extend the interface to
use a different root directory.
Matthias Kruk [Mon, 9 Jan 2023 05:18:39 +0000 (14:18 +0900)]
toolbox.sh: Don't require all interface methods to be implemented
The interface/implements mechanism requires a module to implement
all methods of the interface that it implements, making modules
that extend other modules impossible.
This commit modifies the `implements()' function to remove the
restriction that a module must implement all interface methods.
Matthias Kruk [Mon, 9 Jan 2023 05:13:26 +0000 (14:13 +0900)]
toolbox.sh: Allow modules to implement their own interface
The interface/implements mechanism does not allow a module to
implement methods of the interface that it declares. This makes it
impossible to write modules that extend other modules.
This commit changes the interface mechanism so that a module may
provide methods of the interface that it declares. This is done by
renaming module functions if they clash with names of the interface's
method stubs.
Matthias Kruk [Thu, 22 Dec 2022 06:57:48 +0000 (15:57 +0900)]
toolbox.sh: Don't require interfaces to be included manually
When a module implements an interface, it has to include the module
that defines the interface before calling `implements()'. However,
since the name of an interface and the module that defines it are
necessarily identical, there is no ambiguity and the module should
be included automatically.
This commit modifies the function `implements()' so that it includes
the interface's module before registering the implementation.
Matthias Kruk [Thu, 22 Dec 2022 06:50:18 +0000 (15:50 +0900)]
toolbox.sh: Make interface methods point to error stub by default
Modules that declare an interface are expected to implement all
methods, prefixing method names with an underscore. However, since
there may be (partially) unimplemented interfaces, the framework
should not make such assumptions about implementations.
This commit modifies the implementation of interfaces so that
methods reference an error stub by default, which warns the caller
that the method is not implemented.
Matthias Kruk [Sat, 3 Dec 2022 07:51:59 +0000 (16:51 +0900)]
toolbox.sh: Implement naive interfaces and inheritance
The ipc and uipc modules provide the same API to the module user,
but the user cannot easily switch between the two implementations
because toolbox modules do not have a notion of interfaces.
This commit adds a very primitive means for modules to declare and
implement interfaces: when a module declares an interface, a vtable
is allocated in the form of an associative array, and call stubs
are generated which call the functions referenced in the vtable.
A module that implements an interface can then override entries in
the interface's vtable.
Matthias Kruk [Mon, 9 Jan 2023 04:14:34 +0000 (13:14 +0900)]
test/conf: Add test cases for the conf module
There are no test cases for the conf module, making it impossible to
verify if the module works as intended, and to make sure that commits
do not introduce regressions.
This commit adds a spec file with test cases for the conf module,
adding tests for basic properties of the module's public API.
Matthias Kruk [Sat, 7 Jan 2023 08:38:54 +0000 (17:38 +0900)]
include/json: Make json_array_to_lines() use compact JSON notation
Because the whitespace that is printed by jq when outputting arrays
and objects in non-compact notation does not match the whitespace
that is expected by the test cases, the tests fail.
This commit modifies the `json_array_to_lines()' function so that it
outputs data in compact JSON notation, and changes the test cases so
they do not expect any whitespace at all.
Matthias Kruk [Sat, 7 Jan 2023 08:33:53 +0000 (17:33 +0900)]
include/json: Fix parsing of output from jq in json_array_tail()
The `json_array_tail()' function does not correctly handle return
values and parse output from jq, causing it to incorrectly handle
arrays that contain arrays or objects.
This commit modifies `json_array_tail()' so that it correctly
recognizes the type of data read from jq.
Matthias Kruk [Sat, 7 Jan 2023 07:54:55 +0000 (16:54 +0900)]
include/json: Fix error handling in json_array_head()
Because json_array_head() does not correctly check the return value
from jq, it does not correctly detect if an array is empty.
This commit fixes the function so that it does not rely on the return
value from `jq -e', since it returns an error if the boolean value false
was retrieved.
Matthias Kruk [Sat, 7 Jan 2023 07:25:01 +0000 (16:25 +0900)]
test/{ipc,uipc}: Fix the path to the JSON schema validator
The test specs for the ipc and uipc modules assume to be executed
in the test directory. Thus, they call the JSON schema validator
using a path that is relative to the test directory. However, the
test specs should assume to be executed in the root directory of
the source tree.
This commit changes the paths to the JSON schema validator in the
test specs for the ipc and uipc modules so they are relative to
the root of the source tree.
Matthias Kruk [Sat, 7 Jan 2023 07:10:23 +0000 (16:10 +0900)]
test.sh: Prefer toolbox in same directory over globally installed one
Because test.sh uses the environment's PATH, it ends up using whatever
toolbox is installed in the environment, instead of the toolbox that it
was shipped with.
This commit modifies the test script so that it overrides the path,
making tests use the toolbox they were distributed with. This also
removes unnecessary PATH modifications from the test specs.
Matthias Kruk [Sun, 9 Oct 2022 04:22:47 +0000 (13:22 +0900)]
docs: Add module reference
None of the modules and functions in toolbox are documented. Thus,
users have to read the source code to understand how functions work,
and to find out what functions are there to begin with.
This commit adds a module reference, documenting modules and the
functions that they implement. The following modules are documented
in the reference:
Matthias Kruk [Tue, 27 Dec 2022 06:09:58 +0000 (15:09 +0900)]
include/uipc: Don't store private endpoints in per-user directories
Private endpoints are stored in per-user directories that are created
when the uipc module is loaded. However, the mode and ownership that
the directory is created with may be incorrect, depending on the mode
of the parent directory.
This commit modifies the uipc module so that endpoints are not created
in per-user directories, and moves the necessary directory creation
code to the Makefile, where it will be executed during installation.
Matthias Kruk [Tue, 27 Dec 2022 05:58:47 +0000 (14:58 +0900)]
include/ipc: Don't store private endpoints in per-user directories
Private endpoints are stored in per-user directories that are created
when the ipc module is loaded. However, the mode and ownership that
the directory is created with may be incorrect, depending on the mode
of the parent directory.
This commit modifies the ipc module so that endpoints are not created
in per-user directories, and moves the necessary directory creation
code to the Makefile, where it will be executed during installation.
Matthias Kruk [Tue, 27 Dec 2022 05:53:23 +0000 (14:53 +0900)]
test: Set default GPG key when testing ipc module
When initializing the environment for tests of the ipc module, the
default GPG key is not explicitly set. Because newer GPG versions do
not automatically set the default key, no default key is set, causing
sign operations to fail.
This commit modifies the test cases for the ipc module so that the
default GPG key is set during the test setup.
The default key is not set automatically on newer GPG versions
Matthias Kruk [Tue, 27 Dec 2022 05:52:20 +0000 (14:52 +0900)]
test: Use toolbox in parent directory for tests
Because none of the test specs set PATH, tests use the toolbox version
that is installed in the environment, not the one that is being tested.
This commit modifies the test specs so they prepend the parent directory
to PATH. Further, this adds a symbolic link to the includes in the parent
directory, making sure the correct includes path is searched first.
Matthias Kruk [Mon, 26 Dec 2022 05:10:04 +0000 (14:10 +0900)]
include/uipc: Adjust permissions to allow inter-user IPC
Because the sgid bit is not set on the uipc directory, endpoints do
not inherit the group ownership from the parent directory. Thus,
endpoints are created with a user's primary group instead of the IPC
group, making inter-user IPC communication impossible.
This commit modifies the uipc module and the post-install script of
the debian package so that the sgid bit is set on the uipc directory
and endpoints are created with the correct permissions for uipc
communication between users.
This commit also modifies the Makefile so that the uipc directory is
created during toolbox installation.
Matthias Kruk [Mon, 26 Dec 2022 04:18:22 +0000 (13:18 +0900)]
Makefile,README: Use ':' to separate user and group in chown calls
The use of '.' to separate the user and group in calls to chown has
been deprecated for a while, and newer chown versions now print a
warning if '.' is used instead of ':'.
This commit updates all chown calls in toolbox to use ':'.
Matthias Kruk [Mon, 26 Dec 2022 03:55:32 +0000 (12:55 +0900)]
include/opt: Don't treat options with values as arrays
Because the bit pattern __opt_flag_is_array has more than one set
bit, it is not enough to use a bitwise and to check if an option is
an array, but that is what `opt_print_help()' does.
This commit modifies `opt_print_help()' so that it checks whether
the result of the bitwise and equals __opt_flag_is_array, avoiding
any option with a value to be treated as an array.
Matthias Kruk [Thu, 22 Dec 2022 05:04:54 +0000 (14:04 +0900)]
toolbox.sh: Add the script path to the module search paths
Toolbox requires modules to be located either in ~/.toolbox/include
or in /usr/share/toolbox/include. This is somewhat inconvenient when
developing scripts with modules that are not supposed to be shared
with other scripts.
This commit adds the path of the executing script to the module
search path, allowing script-private modules to be placed in a
directory called "include" next to the script.
Matthias Kruk [Thu, 22 Dec 2022 04:38:06 +0000 (13:38 +0900)]
toolbox.sh: Don't mask realpath's return value
When determining the location of toolbox.sh, the result from realpath
is passed to dirname to remove the script name from the path. However,
both commands are executed in the same statement, causing the return
value of realpath to be masked by dirname.
This commit splits the statement into separate steps, so that return
values are not masked.
Matthias Kruk [Sat, 17 Dec 2022 04:50:31 +0000 (13:50 +0900)]
include/opt: Add array options
Options with multiple values stored in an array can be implemented
using callbacks, but it is somewhat hacky and would be nicer if the
opt module had explicit support for options with multiple values.
This commit adds the "a" attribute, allowing options to be declared
as arrays. The default value of an array option must be the name of
the array that values shall be appended to.
Matthias Kruk [Sat, 3 Dec 2022 07:17:28 +0000 (16:17 +0900)]
include/inst: Silence shellcheck if unused variables are parameters
When scanning the `inst' module with shellcheck, warnings are issued
about unused variables. However, the unused variables are function
parameters, which have to be declared per toolbox convention.
This commit adds comments instructing shellcheck to ignore the
unused function parameters.
Matthias Kruk [Sat, 26 Nov 2022 03:41:25 +0000 (12:41 +0900)]
include/inst: Allow terminated instances to be cleaned up properly
The signal handler that is registered by `_inst_run()' catches the
EXIT and QUIT signals, which causes the handler to be executed one
more time before the process ends. The semaphore that the handler
will attempt to manipulate at that point will have already been
removed, causing the handler to end up in an infinite loop.
This commit changes the trap call so that the EXIT and QUIT signals
are not handled, preventing the signal handler from being executed
more than once.
Matthias Kruk [Sat, 19 Nov 2022 11:49:03 +0000 (20:49 +0900)]
include/ipc: Adjust permissions to allow inter-user IPC
Because the sgid bit is not set on the IPC directory, endpoints do
not inherit the group ownership from the parent directory. Thus,
endpoints are created with a user's primary group instead of the IPC
group, making inter-user IPC communication impossible.
This commit modifies the IPC module and the post-install script of
the debian package so that the sgid bit is set on the IPC directory
and endpoints are created with the correct permissions for IPC
communication between users.
Matthias Kruk [Sat, 19 Nov 2022 10:49:41 +0000 (19:49 +0900)]
include/queue: Create data file when queue is created
The data file that contains the queue data is not created when a
queue is created, but rather when data is inserted into the queue.
This causes issues in applications that use queues to share between
users because they need so set permissions on the entire queue at
the time of initialization. However, because the data file is not
created at that time, its permissions cannot be set.
This commit modifies `queue_init()' so that the data file is created
together with the queue.
Matthias Kruk [Sat, 19 Nov 2022 10:44:16 +0000 (19:44 +0900)]
include/queue: Base64-encoded data in queues
Queues are line-oriented, but the queue functions do not make sure
that user data does not contain newlines, causing data corruption
if the user inserts data that contains newlines.
This commit modifies the queue to store data in base64 format,
avoiding data-dependent behavior.
Matthias Kruk [Sat, 19 Nov 2022 10:38:00 +0000 (19:38 +0900)]
include/queue: Turn queue into a simple FIFO
The queue module contains functions for file queues and queues with
unique contents, which should be used into separate modules to keep
modules simple and avoid mistakes such as using functions for normal
queues on file-queues.
This commit removes functions for file and unique queues from the
queue module, turning queues into simple FIFO data structures.
Matthias Kruk [Sat, 19 Nov 2022 04:34:26 +0000 (13:34 +0900)]
include/inst: Move instance root to /tmp
Instance data is currently kept in the user's home directory, which
has the side-effect that data from crashed instances is not removed
when the machine reboots. Such instances will be shown by `--list',
and prevent singleton instances to be started.
This commit addresses the issue by moving the instance root to /tmp,
which *should* be a non-persistent filesystem.
Matthias Kruk [Sat, 19 Nov 2022 04:23:51 +0000 (13:23 +0900)]
include/inst: Store the instance root directory in a variable
The directory that all instance directories are contained within is
not kept in a variable, causing several functions to construct the
path from TOOLBOX_HOME. Hence, the root directory is effectively
declared in more than one place.
This commit introduces the global variable `__inst_root', which
points to the directory that contains all instance data, making it
unnecessary to construct the path from TOOLBOX_HOME.
Matthias Kruk [Sun, 9 Oct 2022 03:41:17 +0000 (12:41 +0900)]
README: Use uipc in PubSub example instead of ipc
The readme uses the ipc module to explain how to implement simple
pubsub-based messaging using toolbox. However, the ipc module is
somewhat awkward to configure because it uses GPG and requires the
user to generate unencrypted keys.
This commit modifies the readme to introduce the uipc module instead
of the ipc module, since the former is easier to set up.
Matthias Kruk [Sat, 8 Oct 2022 08:24:47 +0000 (17:24 +0900)]
include/mutex: Don't write to stderr when mutex_unlock() fails
When a mutex could not be unlocked, the rm call in mutex_unlock()
writes an error message directly to stderr. However, the function
should only return an error without writing to stderr.
This commit modifies mutex_unlock() so that it does not write to
stderr.
Matthias Kruk [Sat, 8 Oct 2022 08:21:45 +0000 (17:21 +0900)]
include/wmutex: Don't write to stderr when wmutex_unlock() fails
When a wmutex could not be unlocked, the rm call in wmutex_unlock()
writes an error message directly to stderr. However, the function
should only return an error without writing to stderr.
This commit modifies wmutex_unlock() so that it does not write to
stderr.
Matthias Kruk [Sat, 8 Oct 2022 07:38:57 +0000 (16:38 +0900)]
utils/ipc-ssltunnel: Initialize global variables
In ipc-ssltunnel, global variables are used to store the topics and
hooks that are passed to ipc-tap and ipc-inject. These variables are
declared in main(), however not all of them are initialized. While
this is not a bug, the function should either initialize all of the
arrays or none at all.
This commit modifies ipc-ssltunnel to initialize all of the arrays
in main().
Matthias Kruk [Sat, 8 Oct 2022 06:36:51 +0000 (15:36 +0900)]
utils/ipc-sshtunnel: Merge local_to_remote() and remote_to_local()
Except for the position of the call to ssh, the local_to_remote()
and remote_to_local() functions in ipc-sshtunnel are completely
identical.
This commit merges the two functions into one. The new function
expects the caller to pass the direction of the tunnel to be
established in the first argument.
Matthias Kruk [Sat, 8 Oct 2022 06:18:15 +0000 (15:18 +0900)]
utils/ipc-sshtunnel: Allow hooks to be passed on the command line
The ipc-tap and ipc-inject utility scripts allow hooks to be passed
on the command line that implement custom behavior upon receipt of
certain topics or tags. However, because ipc-sshtunnel does not take
hooks as command line arguments, this functionality cannot be used
with tunnels established using ipc-sshtunnel.
This commit modifies ipc-sshtunnel so that users may specify hooks
that will be passed on to ipc-tap and ipc-inject.
Matthias Kruk [Sat, 8 Oct 2022 04:41:40 +0000 (13:41 +0900)]
utils/ipc-sshtunnel: Add script for establishing IPC-over-SSH tunnels
This commit adds ipc-sshtunnel, a utility script for establishing a
bi-directional tunnel for IPC PubSub messages over SSH. This allows
scripts on separate machines to communicate even without a shared
file system.
Matthias Kruk [Thu, 6 Oct 2022 06:21:44 +0000 (15:21 +0900)]
toolbox.sh: Suppress errors in command_not_found_handle()
When the user enters an unknown command, or a script contains an
unknown command, the command_not_found_handle() function provides
suggestions if the unknown command appears to be in a module.
However, because the function does not suppress errors when looking
for modules in the module search paths, it emits errors if any of
the search paths do not exist.
This commit makes the command_not_found_handle() function suppress
errors if any of the module search paths do not exist.
Matthias Kruk [Sun, 2 Oct 2022 05:53:59 +0000 (14:53 +0900)]
utils/ipc-inject: Don't inject messages with header
When ipc-inject injects a message, it uses ipc_endpoint_publish()
to re-publish it in its entirety, including the wrapper it was
received in, causing it to be double-wrapped.
This commit modifies ipc-inject so that it re-publishes only the
data of the received message, instead of the entire message.
Matthias Kruk [Sat, 1 Oct 2022 12:47:33 +0000 (21:47 +0900)]
utils: Implement hooks in ipc-tap and ipc-inject
To correctly forward foundry messages using ipc-tap and ipc-inject,
it is necessary to move data that is usually not transferred over
IPC messages. In order to transfer this data over ipc-tap and
ipc-inject, hooks are necessary that allow user-data to be generated
on the ipc-tap side and consumed on the ipc-inject side.
This commit implements a simple mechanism that allows the user to
specify hooks that are called by ipc-tap whenever a message on a
user-defined topic is encountered. The hook can then inject
messages into ipc-tap's data stream. On the ipc-inject side, the
user may define hooks that are called when a user-message with a
application-defined tag is encountered, allowing the injected
message to be handled on the ipc-inject side.
Matthias Kruk [Sat, 1 Oct 2022 06:02:45 +0000 (15:02 +0900)]
utils: Add interceptor and injector for IPC PubSub messages
Because the IPC module uses the file system to exchange messages,
remote systems cannot communicate without a shared file system such
as NFS. However, NFS may not be an option due to resource and
security constraints.
This commit adds the ipc-tap and ipc-inject utilities, which can be
used to bridge messages over an SSH connection. As the name suggests,
the ipc-tap utility taps into topics and outputs messages published
on those topics to standard output.
The ipc-inject reads messages from standard input and injects them
into local PubSub topics.
include/sem: Implement sem_trywait() using sem_wait() with 0 timeout
Because sem_trywait() does not unlock the waitlock if the counter is
greater than 1, the next caller will be left waiting until sem_post()
is executed.
This commit changes sem_trywait() to call sem_wait() with a timeout
of 0 seconds, since this issue does not affect sem_wait().
include/uipc: Add "topic" field to pubsub UIPC messages
Messages that were sent with uipc_endpoint_publish() don't contain
information about the topic that they were published on, making it
impossible for the receiver to determine what topic a message was
published on if they are subscribed to more than one topic.
This commit adds a new field "topic" to UIPC messages and modifies
uipc_endpoint_publish() so that it sets the field appropriately. This
further adds the function uipc_msg_get_topic() that may be used to
determine the topic a UIPC message was published on.
include/ipc: Add "topic" field to pubsub IPC messages
Messages that were sent with ipc_endpoint_publish() don't contain
information about the topic that they were published on, making it
impossible for the receiver to determine what topic a message was
published on if they are subscribed to more than one topic.
This commit adds a new field "topic" to IPC messages and modifies
ipc_endpoint_publish() so that it sets the field appropriately. This
further adds the function ipc_msg_get_topic() that may be used to
determine the topic an IPC message was published on.
test/ipc: Fix comparisons in test cases for ipc_endpoint_recv()
In the test cases for ipc_endpoint_recv(), the comparisons used to
check the correctness of the received data are inverted, causing the
tests to fail when they should succeed and vice-versa.
test/ipc: Fix return values in test case for ipc_endpoint_open()
When testing whether ipc_endpoint_open() correctly opens a private
endpoint, the return values of the test case are inverted, causing
the test to fail when it should succeed and vice-versa.
This commit inverts the return values of the test case.
Matthias Kruk [Thu, 11 Aug 2022 18:21:40 +0000 (03:21 +0900)]
include/opt: Add a default regex and action
The regex and action of an option may be unset, making it necessary
to check their value before using them.
This commit modifies the opt module to define a default regex and
action in case the user didn't provide one, allowing the parser to
omit checks if the two are unset.
Matthias Kruk [Thu, 11 Aug 2022 18:14:19 +0000 (03:14 +0900)]
include/opt: Initialize value of options that don't have a value
The opt module uses the value of options that aren't followed by a
parameter to count how often the option was on the command line.
However, in this case it never initializes the value.
This commit modifies the opt module so that it initializes the
value of options that are not followed by a parameter to 0.
Matthias Kruk [Thu, 11 Aug 2022 18:04:19 +0000 (03:04 +0900)]
include/opt: Remove summary of short options from help text
The help text displayed by the opt module contains a summary of all
short options in the title. However, a cursory search showed that
most commandline tools don't do that. Thus, this "feature" may be
removed.
This commit removes the summary of short options from the title of
the help text.
Matthias Kruk [Thu, 11 Aug 2022 17:57:48 +0000 (02:57 +0900)]
include/opt: Improve code readability
There are several functions in the opt module where local variables
are not declared in the most narrow block possible, statements are
needlessly spread over multiple lines, and values used only once
are assigned to variables; all of which impeding readability.
This commit modifies the opt module to remove cases of assign-once-
read-once variables, remove integer increments on separate lines,
make the "scope" of variables narrower, and declare variables
with a type, where possible.
Matthias Kruk [Thu, 11 Aug 2022 17:48:34 +0000 (02:48 +0900)]
include/opt: Remove unnecessary array __opt_long
The data that is stored in opt's __opt_long array is a subset of
the data stored in __opt_map. The operations that the module does
on __opt_long might just as well be performed on __opt_map without
affecting the behavior of the module.
This commit removes the __opt_long array from the module and
changes the affected functions to use __opt_map instead.
Matthias Kruk [Thu, 11 Aug 2022 17:46:52 +0000 (02:46 +0900)]
include/opt: Show default values in help text
The opt module does not show the default values of options in the
help text, leaving the user guessing what the values of options are
if they are not specified on the command line.
This commit modifies `opt_print_help()` so that it displays the
default values of options that have a value and a default set.
Matthias Kruk [Thu, 11 Aug 2022 17:09:07 +0000 (02:09 +0900)]
include/opt: Use associative array to track missing options
The opt module does not track missing options, but instead checks
on demand if required options are missing by inspecting all options'
flags and value. This approach is needlessly complicated and slow.
This commit changes the opt module to track required options using
an associative array, allowing the check for missing options to be
performed in constant time.
Matthias Kruk [Thu, 11 Aug 2022 16:54:00 +0000 (01:54 +0900)]
include/opt: Use an associative array to simplify flag parsing
When parsing the flags of an option, the opt module uses a case
construct, which looks clumsy, isn't very easy to read, and makes
adding new flags more complicated than need be.
This commit modifies the opt module to use an associative array
to simplify flag parsing. This way, the parser only has to use a
flag as index into the array in order to find out its internal
value.
Matthias Kruk [Thu, 11 Aug 2022 13:06:36 +0000 (22:06 +0900)]
include/opt: Make return values of opt_get() more helpful
The function `opt_get()` returns 0 when an option is valid but was
not set. This makes it impossible to distinguish if the value of
the option is the empty string or it does not have a value at all.
Further, it does not return an error if an invalid option's value
was requested.
This commit fixes the behavior of `opt_get()` so that it returns 0
if an option is valid and has a value (default or user-defined), 1
in case that the option name is invalid, and 2 in case that the
option name is valid but it does not have a value.