Computing on Towards Freedom

KDE

Sun, 26 Jul 2026 17:33:47 -0700

Various KDE matters are presented here.

Plasma Global Shortcuts Reference

From ~/.config/kglobalshortcutsrc arranged nicely by Qwen AI.

🗂️ Activities

Action	Shortcut
Switch to activity “Action”	`Meta+Ctrl+Shift+A`
Switch to activity “Clj”	`Meta+Ctrl+Shift+J`
Switch to activity “Dac”	`Meta+Ctrl+Shift+E`
Switch to activity “Invest”	`Meta+Ctrl+Shift+C`
Switch to activity “Music”	`Meta+Ctrl+Shift+M`
Switch to activity “NamItAll”	`Meta+Ctrl+Shift+B`
Switch to activity “R1”	`Meta+Ctrl+Shift+Num+End`
Switch to activity “R2”	`Meta+Ctrl+Shift+Num+Down`
Switch to activity “Tweak”	`Meta+Ctrl+Shift+Y`
Switch to activity “Zsh”	`Meta+Ctrl+Shift+N`
Switch to activity “Pnars”	`Meta+Ctrl+Shift+L`

🚀 Application Launchers

Application	Shortcut
Dolphin (File Manager)	`Ctrl+Shift+Return`
Emacs	`Meta+Ctrl+.`
Emacs Client	`Meta+Return`
Falkon (Browser)	`Meta+Alt+U`
Firefox ESR	`Meta+Alt+Return`
Foot (Terminal)	`Meta+Z`
KCalc (Calculator)	`Meta+Alt+T` / `Calculator`
Screen Capture Script	`Meta+Shift+Return`

🪟 Window Management (KWin)

Action	Shortcut
Activate Window Demanding Attention	`Meta+Ctrl+A`
Close Window	`Alt+F4`
Disable Active Input Capture	`Meta+Shift+Esc`
Kill Window	`Meta+Ctrl+Esc`
Maximize Window	`Meta+PgUp`
Minimize Window	`Meta+PgDown`
Minimize All Windows	`Meta+Shift+D`
Move Mouse to Center	`Meta+F6`
Move Window to Next Screen	`Meta+Shift+Right`
Move Window to Previous Screen	`Meta+Shift+Left`
Quick Tile Window (Bottom/Left/Right/Top)	`Meta+Down` / `Left` / `Right` / `Up`
Window Operations Menu	`Alt+F3`
Zoom In / Out / Actual Size	`Meta++` / `Meta+-` / `Meta+0`

🧱 Tiling & Zones (Krohnkite & KZones)

Action	Shortcut
KZones
Activate layout 1 through 9	`Meta+Num+1` to `Meta+Num+9`
Cycle layouts (Forward/Reverse)	`Ctrl+Alt+D` / `Ctrl+Alt+Shift+D`
Move window to next/prev zone	`Ctrl+Alt+Right` / `Ctrl+Alt+Left`
Move window to zone 1 through 9	`Ctrl+Alt+Num+1` to `Ctrl+Alt+Num+9`
Snap active / all windows	`Meta+Shift+Space` / `Meta+Space`
Switch window in current zone	`Ctrl+Alt+Up` / `Ctrl+Alt+Down`
Toggle zone overlay	`Ctrl+Alt+C`
Krohnkite
Focus Down/Left/Prev/Right/Up	`Meta+J` / `H` / `\` / `K`
Grow/Shrink Height & Width	`Meta+Ctrl+J/K/H/L`
Increase/Decrease Size	`Meta+I`
Move Window Down/Left/Right/Up	`Meta+Shift+J` / `H` / `L` / `K`
Next / Previous Layout	`Meta+\` / `Meta+
Toggle Float / Float All	`Meta+F` / `Meta+Shift+F`
Monocle Layout	`Meta+M`

🖥️ Desktop & Workspace Navigation

Action	Shortcut
Show Activity Switcher	`Meta+Q`
Walk through activities (Forward/Reverse)	`Meta+;` (or `Meta+Tab`) / `Meta+Shift+A`
Switch Desktop (Down/Up/Left/Right)	`Meta+Ctrl+Down` / `Up` / `Left` / `Right`
Switch Desktop Left/Right (Alt)	`Meta+Ins` / `Meta+Del`
Switch to Desktop 1 through 6	`Ctrl+F1` through `Ctrl+F6`
Move Window to Desktop (Down/Up/Left/Right)	`Meta+Ctrl+Shift+Down` / `Up` / `Left` / `Right`
Toggle Desktop Grid	`Meta+F8`
Toggle Overview	`Meta+W`
Peek at Desktop (Show Desktop)	`Meta+D`
Present Windows (Current/All/Class)	`Ctrl+F9` / `Ctrl+F10` / `Ctrl+F7`

⌨️ Window Switching & Navigation

Action	Shortcut
Switch to Window (Down/Left/Right/Up)	`Meta+Alt+Down` / `Left` / `Right` / `Up`
Walk Through Windows (Forward/Reverse)	`Alt+Tab` / `Alt+Shift+Tab`
Walk Through Windows of Current App	Alt+`` / Alt+~`

🔊 Audio, Media & Power

Action	Shortcut
Audio & Media
Decrease/Increase Volume (Small)	`Shift+Volume Down` / `Shift+Volume Up`
Decrease/Increase Volume	`Volume Down` / `Volume Up`
Mute Microphone	`Microphone Mute` / `Meta+Volume Mute`
Mute Volume	`Volume Mute`
Media Next/Previous/Play/Pause/Stop	`Media Next` / `Previous` / `Play` / `Stop`
Power & Hardware
Decrease/Increase Screen Brightness	`Monitor Brightness Down` / `Up`
Decrease/Increase Brightness (Small)	`Shift+Monitor Brightness Down` / `Up`
Hibernate / Suspend / Power Off	`Hibernate` / `Sleep` / `Power Off`
Switch Power Profile	`Battery` / `Meta+B`
Toggle Keyboard Backlight	`Keyboard Light On/Off`

🖱️ Touchpad, Tablet & System

Action	Shortcut
Touchpad
Disable / Enable / Toggle Touchpad	`Touchpad Off` / `Touchpad On` / `Touchpad Toggle`
Wacom Tablet
Map to fullscreen / screen 1 / screen 2	`Meta+Ctrl+F` / `1` / `2`
Next / Previous Profile	`Meta+Ctrl+N` / `Meta+Ctrl+P`
Toggle screen map selection	`Meta+Ctrl+M`
Toggle stylus mode / touch tool	`Meta+Ctrl+S` / `Meta+Ctrl+T`
System & Plasma
Activate Application Launcher	`Meta` / `Alt+F1`
Activate Task Manager Entry 1 through 9	`Meta+1` through `Meta+9`
Lock Session	`Meta+L` / `Screensaver`
Show Clipboard Items at Mouse Pos	`Meta+V`
Show Logout Screen	`Ctrl+Alt+Del`
Stop Current Activity	`Meta+S`
Suspend Compositing	`Alt+Shift+F12`
Toggle Screen Reader	`Meta+Alt+S`
Open/Retract Yakuake	`F12`

Musescore

Fri, 24 Jan 2025 13:00:02 -0800

Musescore4 is the most highly rated software for creating notation. It is pretty close to even Lilypond, but offers considerably greater flexibility. A nice bonus is that you can export to musicxml and then use Lilypond’s musicxml2ly (-l english)script to convert!

implode

A midi import left piano part in 2 upper and 2 lower staves. Select both uppers and implode them into a single multivoice staff. Then select that staff and implode again to combine the voices into a single voice. Repeat for the lower staves.

raggedy edge effect

Format > Page settings and play with staff space values.

hide empty bars

Format > Style > Score add hide empty staves rem don’t hide empty staves in first system

Vivaldi

Sat, 27 Jul 2024 20:48:23 -0700

A browser that is powerful, personal and private, adapting to you, not the other way around.

This is the best browser we have ever used. It is full of features and makes one work very effectively!

reading list

Go to reading list button in side panel and add current page using +. (Better than having it on the address bar) this is a very fast way to mark a webpage from there we can do bookmark or omnivore.

cannot enter nickname

The problem appears to be there is something in the bookmarks trash that is getting in the way. After emptying the trash, it accepts the nickname.

toggle extensions

When we hide an extension button, the toggle extension button appears (so we can view the hidden item again). If we remove the button, there is no toggle button.

tab stacking

use any method to stack tabs large numbers use shift key and left click for stacking

save as session (file > save open tabs as session) reload stack from File > Open Saved Session reorder stack using window panel

exports

file > export bookmarks (more intelligent than looking in bookmarks) file > export reading list not working use F2 export reading list

issue with systemd v249

<2021-07-31 Sat> affects all chromium based browsers

one solution was to disable systemd-resolve, but then need to setup /etc/resolv.conf instead of linking it to /run/systemd/resolve/stub-resolv.conf

the other was to downgrade to an earlier version of vivaldi using downgrade from aur, until archlinux makes v249 work properly with chromium again

lack of suitable h264 error

pacman -S vivaldi-ffmpeg-codecs

profile

When we changed the name of schnell to admins, vivaldi locked the profile and wouldn’t start. After some semi-helpful consultation with bing we did: cd .config/vivaldi rm Default/LOCK rm SingletonLock (which pointed to schnell - 1066)

sync

sync help after sync is setup as per help instructions (including encryption key), it seems to happen automatically though on a remote device we need to close and open vivaldi. the remote has to be logged in and have the same encryption key of course.

sort bookmarks

use sidebar to sort bookmarks to sort bookmark bar, right click in empty area and select sort

pdf viewing

not rendering at all difficult to know what the problem is, but it works fine if we reboot … and probably if we restart x too.

dark theme

A fair bit of setting up went into getting the dark theme to function adequately with vivaldi because forcing it messed up certain websites like amazon.ca, so we had to either toggle into white theme (M-]), or adjust the darkness for specific sites using Userstyles. The full discussion is available on Vivaldi dark theme issue: Blacked-out images on forced dark websites thread.

this may be a good way to get into mastedon

Rosegarden

Sat, 27 Jul 2024 20:22:30 -0700

Rosegarden is a music composition and editing environment based around a MIDI sequencer that features a rich understanding of music notation and includes basic support for digital audio. Most important, it can produce lilypond files.

This is likely the best composing option we have beyond plain text entry. One major advantage it offers is the ability to insert bars when there are multiple voices which both denemo and musescore can do, but is laborious with lilypond code. Ideally, we should be able to work with rg and lilypond (via emacs | frescobaldi)

The rg input and playback systems are excellent and there is also a notation view. However, the actual conversion to lilypond may not be as good as we’d like it to be.

EAN!!! Experiment As Needed!!!

intro info

most of the items here are from the rgwiki purpose is to serve as the sequencer of choice for users who prefer to work with music notation. provides a segment-based mechanism for arranging blocks of MIDI and audio data on a canvas tutorials, faq

multiple voices

Do in seperate segments, change the stem orientations (up|down), put onto same track. However, when this converts to lilypond, rg forces separate voices in the code. While the final product looks just right, the ly code is very difficult to work with on it’s own as well as trying to migrate it into lynew.

Therefore, it seems best to just get the other voices to appear in the same segment even though you’ll get ties. Then these can be voiced by ly requirements - may be design a script to do it. Approaching the issue this way is also much simpler as far as inputting goes.

arpeggio symbol

Can be done easily by using T button and then Arp as a ly directive. The result is quite good!

installation components

fluidsynth ln -s FluidR3_GM.sf2 default.sf2 (usr/share/soundfonts) setup /etc/conf.d/fluidsynth (soundfont and alsa daemon) aconnect -o aplaymidi -p128:0
jack2
pipewire pipewire-alsa, pipewire-jack and things work!

fluidsynth gave no latency, but kept crashing. timidity didn’t crash but lagged, so we looked at libao with -OO latency with midi keyboard, timidity++ and rosegarden

here is the solution i finally found to my problem. /usr/lib/systemd/user/timidity.service contains the lines [Service] ExecStart=/usr/bin/timidity -iA so i just need to add on -OO to this and it works by putting me in libao! however, it didn’t enable from ~/.config/systemd/user/ giving Unit ~/.config/systemd/user/timidity.service is added as a dependency to a non-existent unit multi-user.target

rg without softsynth

Haven’t been able to get it to work just just soundfont or linuxsampler.

Not possible as per Bing:

No, you cannot use a soundfont directly in Rosegarden without fluidsynth or another softsynth. Rosegarden is a MIDI sequencer, not a synthesizer. It can only send MIDI data to an output device, but it cannot generate sound by itself. You need a software or hardware synthesizer to play the MIDI data and produce sound¹.

A soundfont is a file that contains samples of real instruments, such as piano, guitar, violin, etc. A soundfont can be used by a software synthesizer, such as fluidsynth, to play the samples according to the MIDI data. A software synthesizer can also use other methods of synthesis, such as additive, subtractive, FM, etc., to create sounds from scratch².

Fluidsynth is a software synthesizer that can use soundfonts or other synthesis methods to generate sound from MIDI data. It can run as a standalone application or as a DSSI plugin in Rosegarden. You can load different soundfonts into fluidsynth and select different instruments for each track in Rosegarden³.

If you want to use a soundfont directly in Rosegarden without fluidsynth, you will need a hardware synthesizer that can load soundfonts, such as the Sound Blaster series of soundcards. You will need to connect the hardware synthesizer to your computer via a MIDI interface and configure Rosegarden to use it as an output port. You will also need to load the soundfont into the hardware synthesizer using a utility program, such as sfxload⁴.

I hope this explains why you need fluidsynth or another softsynth to use a soundfont in Rosegarden. 😊

Source: Conversation with Bing, 9/5/2023 (1) frequently_askedquestions | Rosegarden Wiki. https://www.rosegardenmusic.com/wiki/Frequently_Asked_Questions. (2) Improving Your Linux System For Creating Music - Rosegarden: music. https://www.rosegardenmusic.com/tutorials/qsynth-rosegarden/index.html. (3) LinuxQuestions.org - [SOLVED] Rosegarden, FluidSynth, and JACK refuse …. https://www.linuxquestions.org/questions/slackware-14/rosegarden-fluidsynth-and-jack-refuse-to-work-4175420532-print/. (4) using_rosegardenwithqsynthlookmanojackd | Rosegarden Wiki. https://www.rosegardenmusic.com/wiki/using_rosegarden_with_qsynth_look_ma_no_jackd.

rg with timidity

using qpwgraph, crackling sounds are sometimes heard probably due to some feedback loop if usb audio device (or whatever) capture mono -> rosegarden record in 1 L disconnecting this seems to work and crackling disappears (sometimes even if we reconnect)

rg with yoshimi

a minor issue is that unlike timidity, yoshimi isn’t playing the very first note (though the sounds come out much cleaner). so we should probably always have a blank bar. printouts work fine though since the blankbar is ignored.

all this in the post works, but yoshimi needs to be run in floating mode, otherwise the changes in rg won’t come through. or just start it in cli.

the numbers of the instruments are changed, but the banks aren’t … yet

yoshimi.rgd

this has to go to ~/.local/share/rosegarden/library/, but the one we downloaded doesn’t match with the actualy instruments. so we mv yoshimi.rgd yoshimi.rgd.gz gunzip yoshimi.rgd.gz and edit it.

one issue is updating the id numbers for The_MysteriousBank and since sed doesn’t do non-greedy, use perl perl -pe ’s/id="(+̣)"/“id=\”.($1+1)."\"/e’ tt.tt not that we really understand this too well - we require some lang may be clojure, rust … to make such manipulations.

rather than do the above we wrote script

#!/usr/bin/env zsh

# creates ~/.local/share/rosegarden/library/yoshimi.rgd from /usr/share/yoshimi/banks
# usage: jri (must use zsh or parameter substitution will not work)

YDIR=/usr/share/yoshimi/banks
yoshimirgd=~/.local/share/rosegarden/library/yoshimi.rgd

# first part of a yoshimi.rgd file
T1='<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE rosegarden-data><rosegarden-data version="4-0.9.1">
    <studio recordfilter="0" thrufilter="0">
    <device direction="play" id="0" name="Yoshimi" type="midi" variation="LSB">
        <librarian email="lorenzofsutton@gmail.com" name="Lorenzo Sutton" />
        <controls>
            <control colourindex="2" controllervalue="10" default="64" description="&lt;none&gt;" ipbposition="0" max="127" min="0" name="Pan" type="controller" />
            <control colourindex="3" controllervalue="93" default="0" description="&lt;none&gt;" ipbposition="1" max="127" min="0" name="Chorus" type="controller" />
            <control colourindex="1" controllervalue="7" default="100" description="&lt;none&gt;" ipbposition="2" max="127" min="0" name="Volume" type="controller" />
            <control colourindex="3" controllervalue="91" default="0" description="&lt;none&gt;" ipbposition="3" max="127" min="0" name="Reverb" type="controller" />
            <control colourindex="4" controllervalue="64" default="0" description="&lt;none&gt;" ipbposition="-1" max="127" min="0" name="Sustain" type="controller" />
            <control colourindex="2" controllervalue="11" default="127" description="&lt;none&gt;" ipbposition="-1" max="127" min="0" name="Expression" type="controller" />
            <control colourindex="4" controllervalue="1" default="0" description="&lt;none&gt;" ipbposition="-1" max="127" min="0" name="Modulation" type="controller" />
            <control colourindex="2" controllervalue="74" default="64" description="&lt;none&gt;" ipbposition="2" max="127" min="0" name="Cutoff Freq" type="controller" />
            <control colourindex="2" controllervalue="71" default="64" description="&lt;none&gt;" ipbposition="2" max="127" min="0" name="Resonance" type="controller" />
            <control colourindex="2" controllervalue="73" default="64" description="&lt;none&gt;" ipbposition="2" max="127" min="0" name="Attack time" type="controller" />
            <control colourindex="2" controllervalue="72" default="64" description="&lt;none&gt;" ipbposition="2" max="127" min="0" name="Release time" type="controller" />
            <control colourindex="4" controllervalue="1" default="8192" description="&lt;none&gt;" ipbposition="-1" max="16383" min="0" name="PitchBend" type="pitchbend" />
        </controls>'

# second part of a yoshimi.rgd file
T2='</device>
</studio>
</rosegarden-data>'


# put part1 on
echo $T1 > $yoshimirgd

# go to /usr/share/yoshimi/banks for further operations
cd $YDIR

# templates to substitute into
Btempl='<bank lsb="0" msb="NUM" name="BANK" percussion="false">'
Itempl='<program category="BANK" id="IDEN" name="NAME" />'


# put the banks and instruments in to the yoshimi.rgd file
N=0
for BA in $(ls -1 | LC_COLLATE=C sort); do
    ((N+=5))
    Btxt=${${Btempl:s/NUM/$N/}:s/BANK/$BA}
    echo $Btxt >> $yoshimirgd
    for IN in $BA/*; do
        #ID=${${IN%%-*}#*/}
        ID=$(echo ${${IN%%-*}#*/} | bc) #str->int
        ((ID-=1)) #dec ID so instruments match up on import
        NAM=${${IN#*-}%.*}
        Itxt=${${${Itempl:s/IDEN/$ID}:s/NAME/$NAM}:s/BANK/$BA}
        echo $Itxt >> $yoshimirgd
    done
    echo "</bank>" >> $yoshimirgd
done

# put the part2 into the yoshimi.rgd
echo $T2 >> $yoshimirgd

echo "all done"

exit

post @folderol

i have things working now pretty nicely between yoshimi and rg.

i downloaded this yoshimi.rgd after reading this thread which addressed an issue i was having with rg resetting anything i did in yoshimi to whatever it felt like (well that’s what it felt like): Rosegarden - Yoshimi issue

i put the yoshimi.rgd into ~/.local/share/rosegarden/library/ where there are a huge number of rgd files for various instruments and synthesizers (too i guess).

then in rg, i went into Studio > Manage MIDI Devices where i

deleted the default general midi bank (though it keeps coming back, hence i have to open a template), because i don’t really need it and never could do anything with it either
created a new device (click New in topleft box), then click Bank, then click Import (bottom left box).

the import button gave me access to the yoshimi.rgd file and eventually i had the banks and instruments showing up in the instrument parameters box from where i could assign them to various tracks.

i found though that i had to set the channel dropdown to ‘fixed’ or all the tracks would change to the same thing when i set a single track. every instrument i set for an rg midi track, is reflected in yoshimi too! (however, that is true only within the bank that is chosen on yoshimi if the instrument number exists - i can’t seem to change the bank in yoshimi from rg - may be we shouldn’t and just gather up the instruments we want into a custom bank?)

i midied out the melody, and a piano accompaniment for the first 4 bars of twinkle, twinkle little start - hey, jane taylor and mozart (even though he didn’t really write the melody) would be pleased. then i recorded audio in rg, singing out both (though not simultaneously) a tenor and baritone part - neither jane nor wolfgang would have been pleased, but rg didn’t crash!! (may be because yoshimi was doing the midi heavy lifting instead of timidity|fluidsynth?)

so, assuming this is the a|the correct way to do these things, all is well indeed!

TODO ornaments

while not as convenient as musescore, we likely have much better control of the ornaments played as midi in rg than lilypond’s articulate.ly, so we should create a file of ornaments that can be used in any composition. this file needs to be loaded first and then we merge the actual composition file to work with.

rg lyrics entry AVOID

compose notes copy lyrics editor template (which contains the note layout nicely) input using emacs (seems to be much faster than using the rg editor) paste emacs work into lyrics editor

only purpose verses serve is if you want same music as with hymns - for praductions, it is likely going to mess everything up!

DONE rg -> ly

some things don’t quite come out as well

we can play with rg
we can copy and paste rg notes into established ly templates
we can modify specific parts of the rgly with a script (probably best solution and DONE!)

tour

Editing
MIDI Configurations for over 100 well-known MIDI synths are included
Notation
Audio let Rosegarden’s sample-accurate audio engine do the hard work -> this might be the issue?
Synths not sure what this is
Integration jack is standard, but we use pipewire-jack and wonder if this is why audio crashes?

handbook

1. Introduction

Rosegarden is a versatile software that functions as both a MIDI and audio sequencer and a musical notation editor.
It allows users to record, arrange, compose music in traditional score, MIDI data, or audio formats.
Users can write, edit, and organize music compositions and add synthesization, effects, and mixdown capabilities.
Rosegarden provides comprehensive notation editing for high-quality printed output.
The software is designed to be user-friendly for both experienced Linux users and newcomers to music software.
The handbook serves as a reference for Rosegarden’s capabilities and concepts and can be used sequentially or as a reference guide.
Users can access tutorials on the Rosegarden website for further assistance.
Rosegarden’s functions are typically accessible through menus, but keyboard shortcuts and toolbar icons are also available for convenience.
Some of Rosegarden’s features are optional, depending on the installation and available libraries.
Dependencies for optional features are documented in the Rosegarden README and configuration documentation, and users should consult their configuration documentation or distributor if certain features are not visible in their version of Rosegarden.

2. Rosegarden Files and Documents

Rosegarden stores compositions in its own internal format with a .rg file extension.
It can also handle various standard file formats.

New compositions

When starting Rosegarden, a new document with an empty composition is created, requiring users to either record or create segments to start editing.
Segments are blank lengths of music that can be filled in using editing tools.
The software’s binary file format is actually gzipped XML, despite complaints about its opacity.

Using Midi Files

Rosegarden provides functions for importing and exporting MIDI files through “File->Import->Import MIDI File” and “File->Export->Export MIDI File.”
While Rosegarden can work with file formats like MIDI, it does not store MIDI data internally.
It’s advisable to save files in Rosegarden’s own format to retain comprehensive information, as exporting to MIDI may result in the loss or rearrangement of certain file structure details.
When loading a MIDI file in Rosegarden, it organizes the music into separate tracks for each MIDI track or channel, establishes instrument mappings, and attempts to generate plausible notation for the MIDI performance data.

Rosegarden Project Files

Rosegarden composition files (.rg) store note information, MIDI controller settings, plugin details, and audio file references for a composition.
The actual audio files are not stored within the composition file, and users are responsible for managing their location on disk.
Rosegarden Project files are helpful for conveniently bundling a composition, its associated audio files, plugin data, and additional files into one package.
To export a composition and its data as a Rosegarden Project file, use “File->Export->Export Rosegarden Project file…”; to import such a file and reconstruct a composition, use “File->Import->Import Rosegarden Project file…”
It’s important to note that Rosegarden Project format is for interchange purposes, not long-term archival, and compatibility may vary between Rosegarden versions.

Other file formats

Rosegarden supports exporting to and importing from various file formats, providing flexibility in working with different music systems.
Csound: Export score files for the Csound synthesis system using “File->Export->Export Csound score file…,” although not all Rosegarden data may be saved in Csound format.
Mup: Save the current composition as a new Mup file using “File->Export->Export Mup file…,” suitable for use with the Mup music publication system, though some adjustments may be needed for quality output.
MusicXML: Use “File->Export->Export MusicXML file…” to save the composition as a MusicXML file, compatible with music software supporting MusicXML, but note that support is experimental and may not preserve all data.
Hydrogen: Rosegarden can import h2song files from the Hydrogen drum machine, creating a MIDI-style structure with one track per pattern. Audio samples are not imported; use “File->Import->Import Hydrogen file…” for this.
Rosegarden 2.1: Create a new composition from an existing X11 Rosegarden v2.1 (.rose) file using “File->Import->Import Rosegarden 2.1 file…” for compatibility.

Merging files

In Rosegarden, you can merge external files into the current document, incorporating their data into the existing composition.
To perform a merge, use the appropriate option found on the File->Merge submenu instead of the Open or Import options.

Printing

Rosegarden offers the ability to print compositions as musical scores.
Printing directly from MIDI data may result in less readable output, so it’s recommended to use the notation editor to tidy up the score before printing.
While Rosegarden is capable of producing readable scores, for true print-quality scores, users should consider using LilyPond in conjunction with Rosegarden.
LilyPond is a dedicated music typesetting software that produces high-quality typeset output from specialized text files.
Rosegarden can export LilyPond files using “File->Export->Export LilyPond,” and it provides access to LilyPond features not available in Rosegarden’s native interface.
The File->Preview with LilyPond function allows for convenient previewing of LilyPond-formatted scores using a PDF viewer.
LilyPond options are available when exporting, previewing, or printing with LilyPond, offering additional customization.

LilyPond options

When using LilyPond export functions in Rosegarden, a dialog allows for specific LilyPond export options.
LilyPond export supports various paper sizes and landscape mode for printing.
Font size settings can be adjusted to control the size of notes and staffs on the printed score.
Users can choose to export content from all tracks, non-muted tracks, the currently selected track, or selected segments.
There’s an option to merge tracks with the same name, useful for creating piano scores with multiple voices.
Tempo marks, lyrics, and beamings can be exported based on user preferences.
Users can adjust the alignment of lyrics as Left, Center, or Right.
The “Ragged bottom” option allows for positioning extra vertical space at the bottom of the page when there are few staffs.
Enabling “point-and-click” debugging allows clicking notes in the PDF output to open the LilyPond source.
LilyPond can produce MIDI output, and users can choose to enable this option.
Headers in LilyPond output can be edited both in the export dialog and through the Composition->Edit Document Properties function.
NewPrinting compositions must either record something, or create some blank lengths of music (referred to as “segments”) that you can then fill in
Using MIDI files it does not store MIDI data internally Rosegarden splits out the music into one track per MIDI track or channel in the file
Rosegarden Project files contains your composition, all of the audio files it uses, any further data files required for plugins used in the composition, and any additional files you may want to include, all in one big bundle.
Other file formats Rosegarden can export to and import from a number of other file formats.
Merging files loading or importing the data from the file, as necessary, and adding it to the existing composition
Printing to produce true print-quality scores you should consider using LilyPond in conjunction with Rosegarden
LilyPond options export All tracks, Non-muted tracks, the current Selected track, or Selected segments Merge tracks that have the same name: for voices Export tempo marks Export lyrics: don’t do lyrics here Export beamings: use lp default Export block Headers: do automatically through lypdnew.sh

3. The Track Editor

Overview

Rosegarden’s layout is centered around a track structure, common to many sequencers.
The main window features a track editor with a list of tracks and their associated instruments displayed on the left side.
Users can manipulate and move blocks of music data referred to as “segments” within the track editor.
Specialized editors like notation, matrix, percussion matrix, and event list can be used to work with segments.
The type of segment created depends on the track it’s placed on; for example, an audio track creates audio segments, and a MIDI track creates MIDI segments.
Track parameters can be set to pre-select segment properties at the track level and fine-tune MIDI recording.
Tracks can be assigned to various MIDI, plugin synth, or audio instruments.
The button area on the left of the segment canvas displays track number, mute and record status, and track labels.
Users can mute/unmute tracks andselect a track as a record target by clicking on the respective buttons.
Track labels are optional and can be displayed via the Settings->Show Track Labels menu option, otherwise, instrument assignments are shown.

Tracks and instruments

Each track in Rosegarden must have an instrument assigned to it to produce sound.
The first 16 tracks are typically assigned to the 16 instruments of the first available MIDI device, while the next 16 tracks are assigned to audio.
To change the instrument assignment for a track, right-click on the track label, and a pop-up menu will allow you to select from available instruments.
Instruments can represent a single channel on a MIDI device or a mono/stereo audio output or audio synth plugin.
MIDI or synth plugin instruments are suitable for tracks containing note data, while audio instruments are used for tracks containing audio.
Multiple tracks can be assigned to the same instrument, causing them to produce sounds in a similar way.
The Instrument Parameter Box allows users to adjust various properties of the instrument associated with a track, such as selecting a specific patch for a MIDI instrument to produce a particular sound.
For more information about instruments, users can refer to the Studio section.

Creating segments

To create an empty segment in Rosegarden, use the pencil tool, which is typically the default active tool when starting Rosegarden.
Click on the pencil icon on the toolbar and then click at the point where you want the segment to start, at the appropriate height for the track and the correct time.
Drag rightwards to define the length of the segment in bars.
Segments can also be created using the Select tool by clicking and dragging with the middle mouse button or the left button with Ctrl held down.
If you need to draw a segment longer than the visible canvas, the canvas will scroll automatically once you reach its edge. [Can also use Segment > Stretch or Squash to assign number of measures.]
Holding Shift while clicking and dragging prevents the snap-to-grid effect.
New segments are initially in yellow color and display a treble clef by default.
Explore track parameters for more segment customization options.
To create a segment that starts in the middle of an existing segment, hold Ctrl and Alt when clicking. The track will expand to accommodate the new segment.
After creating a segment, you can edit it using the matrix, percussion matrix, or notation editors by double-clicking on the segment or right-clicking and choosing the desired editor from the context menu.
When recording from a MIDI device or audio input, new segments are automatically created for each recording.

Audio segments [probably will not use because of lmms]

An audio segment in Rosegarden contains recorded or sampled audio data, not editable note-event data.
Rosegarden enables you to record, play, arrange, and split audio segments but does not offer audio waveform editing features.
You can create an audio segment by recording from an audio source onto a track assigned to an audio instrument or by working with existing WAV files.
To manage audio segments, use the audio manager dialog accessible through File->Manage Audio Files.
You can import audio files by dragging and dropping them into the audio file manager or using its File menu.
To add audio files to a track, either drag-and-drop them onto Rosegarden’s track canvas or use the audio file manager’s Action->Insert into Selected Audio Track menu function if the selected track is assigned to an audio instrument.
Audio files associated with a composition are saved with the composition, but the audio files themselves are not. Maintain a proper audio file search path (via Composition->Edit Document Properties…->Audio) to keep audio files accessible together.
You can export your project to a Rosegarden project package file (.rgp) via File->Export->Export Rosegarden Project File to collect scattered audio files in one place.
The audio manager dialog displays sample-rate and other information for each audio file, marking any with a different sample rate than the current JACK audio session. Rosegarden will resample these files during playback, but you are advised to resample them separately using dedicated audio editing software for better quality.

Manipulating segments

In Rosegarden, you can manipulate segments using various tools available from the main toolbar.
The Select tool serves multiple purposes, including selecting, moving, copying, resizing, and creating segments.
To move a segment, use the Move tool and click and drag it. Holding Ctrl while dragging creates a copy instead of moving.
Resizing a segment is done with the Resize tool, which allows you to extend or shorten a segment. Holding Ctrl while resizing rescales the segment.
Stretching or squashing a segment can be achieved by holding Ctrl while resizing. This affects the speed of MIDI events or the time-stretching of audio.
The Split tool is used to split a single segment into two separate segments. Click on the segment you wish to split with the Split tool.
You can split segments at the closest barline to the cursor by default. Holding Shift while splitting prevents this.
Rosegarden offers additional Split functions in the Segments menu, including Split on Silence, Split by Pitch, Split by Recording Source, and Split at Time, each with specific functions for segment splitting.

Tempo and Time Signature

In Rosegarden, a composition is associated with a single stream of tempo changes and time signature changes.
Tempo changes can be constant or smoothly ramping to a different target tempo at a specific future time.
Tempo is displayed in beats per minute, and its duration depends on the time signature.
The tempo ruler in Rosegarden’s editors shows tempo and time signature changes throughout the composition.
You can adjust tempo changes by clicking and dragging the tempo line in the tempo ruler.
To insert a new tempo change, right-click on the ruler and select “Insert Tempo Change.”
The tempo and time signature editor lists all tempo and time signature changes and allows individual editing.
You can add or edit tempo changes using various methods, including the tempo ruler and the tempo and time signature editor.
Time signature changes in Rosegarden affect the structure of beats in each bar, and you can add or edit them using Composition menu functions or the tempo and time signature editor.
You can set tempo from an existing segment using functions like “Set Tempo to Audio Segment Duration” or “Set Tempos from Beat Segment” for MIDI events. [tempo map: you can record from a MIDI keyboard and tap a series of single notes at approximate beat positions, speeding up and slowing down gradually; then select the recorded segment !!!]

Markers, ranges, and loops

Markers in Rosegarden are moments in time with labels, useful for defining specific points in your composition. [sort of like commenting]
Markers are displayed in the main ruler at the top of Rosegarden’s editor windows.
To add a new marker, double-click on the marker area of the ruler, and you can edit its name and location in the marker editor. [easier to use right-click for insert and edit; also S-m to insert]
Clicking on the text of a marker in the marker ruler moves the playback pointer to that marker’s location.
You can set a loop range between two marker positions by clicking in the area between them with the Shift key pressed.
move to marker prev|next with C-,|.
Rosegarden also features a “quick marker” that can be defined [only one] on the fly with a single key and is useful for jumping to a specific position during recording. [Quick marker can only be moved, not removed, but it isn’t saved either, so it doesn’t matter. C-S-m insert; S-m jump to; C-1,1? !!!]
Ranges and loops allow you to highlight a period of time in the main ruler, used for editing operations and looping during playback.
To set the current range, click and drag with the Shift key pressed on the main ruler or between marker positions in the marker ruler.
Range editing enables you to cut, copy, and paste sections of music within the defined range, including segments, tempos, and time signatures.
You can insert an empty range at the current playback position to create space for composition using the Edit->Insert Range menu function.

Triggered segments

A triggered segment in Rosegarden is a segment that doesn’t appear on the main canvas but is triggered during playback by specific notes in other segments.
Triggered segments are managed through a window accessible via the Segments->Manage Triggered Segments option.
To create a new triggered segment, copy a segment or part of a segment to the clipboard, then use the Edit->Paste as New Triggered Segment menu option in the manager window.
Triggered segments can be stretched or squashed to match the duration of the triggering note, and their pitch and velocity can be adjusted to match the triggering event.
Playback timing options for triggered segments include playing as stored, truncating if longer than the note, ending at the same time as the note, or stretching/squashing to match the note duration.
Triggered segments may contain silence at the start or end, which will be played and adjusted as well.
It’s not possible to make a triggered segment start playing before the start time of its triggering note, trigger a segment from within another triggered segment, or trigger audio segments.
Editing a triggered segment affects all the notes that trigger it, not just the one used for editing. [All this sort of works, but needs experimenting !!!]

4. The Transport

[Much of this is irrelevant now since we are interested in notation - the only part that may be of interest is if we record midi in real-time and quantize.] Overview

The Transport in Rosegarden controls playback and displays the playback pointer’s position.
You can activate the Transport if it’s not visible by going to Settings->Show Transport.
The Transport displays time reached by the playback pointer in seconds by default but can be switched to show musical time (bar and beat counts), musical time with a metronome, or audio sample frame times.
It can toggle the display to show time remaining until the end of the composition.
Main controls on the Transport include starting and pausing playback, fast-forwarding, rewinding, stopping, and jumping to the beginning or end of the composition.
The Transport also shows the tempo and time signature in effect at the current playback position, which can be adjusted by double-clicking on them.
You can enter a precise time for the playback position by double-clicking anywhere on the main time display in the Transport window.
Additional controls and a MIDI event display are available in the bottom flap of the Transport, which can be toggled on or off.
These additional controls include buttons for recording, setting loop start and end points, soloing tracks, controlling metronomes, and silencing MIDI devices (the panic button).

Recording

To record a new segment into a track in Rosegarden, ensure the appropriate tracks are armed for recording by selecting the red (for audio tracks) or yellow (for MIDI tracks) LED-style buttons on the track labels.
Use the Transport’s record control to start the transport, play the existing composition, and simultaneously record into the selected track(s).
Rosegarden allows you to record multiple MIDI and audio tracks simultaneously, provided your computer has enough power and sufficient disk space.
Be cautious not to leave tracks selected when you intend to record in a different location.
Punch-in record, toggled with the Space bar, lets you switch between playback and recording without stopping the transport or arm for recording when the transport is stopped.
When recording to MIDI tracks, the recording source is determined by the MIDI devices selected in the MIDI device manager, filtered through the track level recording filter and global MIDI filters.
For audio tracks, the recording source for each track is the input selected for that track’s instrument in the instrument parameter box, which can be an external audio input or an audio submaster/master.
By default, audio tracks record from the soundcard’s first available physical record input, but you can use a JACK connection tool to connect other JACK audio applications to the input.

Synchronising to external transports

Rosegarden supports the JACK transport mechanism for synchronizing its transport with other JACK applications on the same computer.
To activate JACK transport, go to Settings->Configure Rosegarden… and check the “Use JACK transport” box.
When JACK transport is active, multiple applications subscribed to it share a common transport control, allowing you to start, stop, and move the transport from any of them.
Many Linux audio applications that use the JACK audio server also support JACK transport. Check their documentation for details.
Rosegarden also supports MIDI Time Code (MTC) synchronization in both master and slave modes.
MTC synchronization is useful for synchronizing Rosegarden with hardware MTC-capable devices or software on other computers connected via MIDI.
To set Rosegarden as the MTC Master, go to Settings->Configure Rosegarden… and in the MIDI page’s MIDI Sync tab, select “MTC Master” from the MIDI Time Code menu.
To make Rosegarden an MTC Slave, select “MTC Slave,” and Rosegarden’s transport will automatically synchronize with incoming MTC events.

Infrared Remote Controls

Rosegarden supports infrared remote controls for almost all transport functions.
To use infrared remote controls with Rosegarden, you need either the internal LIRC client (optional feature at compile time) or the KDE IRKick application (KDE LIRC daemon) available in your Linux distribution’s repositories.
You also need to install and configure the LIRC server daemon, with specific details depending on your remote control and receiver.
For more information and details about the internal LIRC client configuration file and IRKick basics, you can refer to Rosegarden’s website and the LIRC website.

5. The Studio

Devices, instruments and connections

Rosegarden refers to everything related to the sound systems attached to the computer as the “Studio.”
The Studio includes information about MIDI and audio hardware, connections, configurations, MIDI patches, and more.
Three essential terms in the Studio context are: Device, Instrument, and Connection.
A “Device” in Rosegarden is something capable of producing sounds, which can be a MIDI or audio device.
An “Instrument” is a sound-producing entity within a device that you can assign tracks to, controlling the sound they produce.
A “Connection” specifies the physical MIDI port a MIDI device is attached to and can be configured in the MIDI device manager.
Rosegarden may create default devices for available connections on startup, but it’s essential to configure them to match the actual hardware setup.
The studio setup is stored with every Rosegarden-format file you save.

Managing MIDI devices

The MIDI device manager dialog in Rosegarden displays available MIDI devices and their connections.
You can access this dialog through Studio->Manage MIDI Devices.
Default device configurations are provided at startup, but you can modify them in this dialog.
“Play devices” in the dialog provide targets for MIDI tracks, with each Play device having 16 instruments to assign tracks to.
By assigning a connection to a Play device, tracks assigned to it will play to the appropriate channel of any MIDI synth connected to that connection.
“Record devices” represent potential sources for recording MIDI, and you can make them current for recording by using the checkboxes in the dialog.

Managing banks and programs for MIDI devices!!!

Each MIDI device in Rosegarden corresponds to a set of instruments, and each instrument corresponds to a channel on your MIDI synth.
You can assign tracks to these instruments and choose which sounds (voices) they produce, but Rosegarden needs to know the names for all available voices.
The dialog for managing banks and programs allows you to attach banks of programs (voice names) to your MIDI devices, enabling you to set each instrument on a device to use one of these voices.
You can import program names from device (".rgd") files or create a new bank manually if there is no existing .rgd file for your synth.
Percussion key mappings are available for percussion programs, and they can be defined in the .rgd file.
For soft-synths or soundcard synths using SoundFont (.sf2) files, program names can be loaded directly from the .sf2 file into the dialog.
Some MIDI devices use a “variations” scheme, and you should enter program names in a bank format. Rosegarden can support devices with variations, and you can use the “Show Variation list based on MSB/LSB” option to display variations instead of banks in an instrument parameter box.
When importing a MIDI file, Rosegarden uses the existing studio setup to map MIDI programs to instruments, so ensure your studio is correctly configured for your devices before importing MIDI files to avoid losing program data.

Audio Routing

Any Rosegarden track assigned to an audio or synth plugin instrument becomes part of Rosegarden’s audio routing system.
There are 16 audio instruments and 16 synth plugin instruments, each with adjustable parameters such as fader level, pan, and effects.
The input connection of an instrument defines where Rosegarden reads audio data from when recording to that instrument. Common settings include external audio inputs, Rosegarden’s built-in audio submasters, or the master output for mixdowns.
The output connection of an instrument defines where the audio goes when played. Instruments can send audio to Rosegarden’s master output or up to eight internal submaster busses for grouping instruments.
Rosegarden supports pre-fader insert effects slots for each audio instrument but does not yet support effects on submaster busses or send effects. Users can achieve similar results using external JACK effects and mixing.
Further details about audio routing, mixing, and effects can be found in the audio mixer, instrument parameter box, and audio plugins sections.

MIDI Routing

Rosegarden features MIDI Thru, which allows for the instant sending of incoming MIDI events to specific output devices and MIDI channels.
MIDI Thru is enabled by default in Rosegarden but can be globally disabled through the menu option Studio->MIDI Thru Routing if you prefer to use an external MIDI routing system.
There is a global filter for MIDI event types that lets you choose which event types to route and which to ignore. Details about this filter can be found in the MIDI filters section.
MIDI Thru routing behaves differently when Rosegarden is playing or stopped compared to when it is recording.
During playback or when Rosegarden is stopped, MIDI Thru routes events to the selected track’s instrument, indicated by the highlighted track name.
During recording, MIDI events are routed to the armed track instruments based on the recording filters. The selected track does not matter; the routing is based on the filters.
Each event corresponds to the first track with a matching record filter.

The Audio Mixer

Rosegarden’s audio mixer window, accessible through Studio->Audio Mixer, resembles a hardware audio mixer.
It allows you to view and modify the state of multiple audio instruments simultaneously.
The audio mixer provides controls such as playback level meters, fader controls, routing options, pan settings, mono/stereo toggles, and effects plugin slots for each audio and synth plugin instrument.
You can also manage the master audio output level, record level, and submaster output levels within the audio mixer.
The Settings menu in the audio mixer is significant and includes audio routing settings for stereo audio record inputs and the number of submasters.
Adjusting the number of submasters explicitly to “None” when they are not in use can lead to a small performance improvement.

The MIDI Mixer

The MIDI Mixer in Rosegarden allows users to view and edit instrument parameter values for multiple MIDI instruments simultaneously.
The window is organized with tabs, one for each playback MIDI device in the studio configuration.
Selecting a device tab reveals controls for the 16 instruments associated with that device.
The displayed controls are based on the settings in the control parameter manager and include volume adjustments (corresponding to MIDI volume control change) and approximate output level indicators.

Audio Plugins

Rosegarden uses audio plugins, which are separate software pieces that can manipulate audio signals in various ways. They are categorized into effects plugins and synth plugins.
Effects plugins alter audio signals, such as reverb, delay effects, or compression.
Each audio or synth plugin instrument in Rosegarden has five associated effects slots, which can be used to chain up to five effects onto the audio signal before it undergoes fader level and pan processing.
To load an effects plugin into a slot, you can click its button on the instrument parameter box or in the audio mixer window.
Synth plugins are used to generate sounds from note event segments and can replace hardware or standalone soft-synth MIDI devices.
Synth plugins provide better control, repeatability, and routing options compared to external MIDI devices.
Synth plugins can be configured by selecting a track, assigning it to a synth plugin instrument, and then loading a synth plugin.
Some plugins may have their own editor windows for more extensive control.
Using plugins can consume CPU resources and may affect reliability, as crashes or hangs in a plugin can impact Rosegarden’s stability.

Metronomes

Rosegarden has two built-in metronomes, one for recording and one for normal playback.
These metronomes send notes to a MIDI instrument and use the same instrument, pitch, and velocity information.
You can configure the metronomes by going to Studio->Manage Metronome, where you can select a MIDI instrument, set the resolution (beat, subdivision, or bar), pitch, and velocities.
The actual voice the metronome uses depends on the instrument selected, similar to how MIDI tracks depend on their assigned instruments.
The record and play metronomes can be enabled or disabled using the metronome button in the Transport window. By default, the record metronome is on, and the play metronome is off.
There’s also a “visual metronome” feature that makes the transport window flash brightly to mark each beat during playback.

MIDI filters

The MIDI filters dialog in Rosegarden allows you to specify which MIDI events should be ignored or accepted by the software.
You can use checkboxes in this dialog to indicate which types of MIDI events you want to filter out or record.
This feature is helpful for customizing how Rosegarden handles MIDI events when forwarding them using MIDI THRU or recording them.

External MIDI control surfaces

Rosegarden can be controlled to some extent by an external MIDI device, allowing you to adjust mixer settings and other features.
To set up external MIDI control, you need to use an ALSA MIDI connection manager like qjackctl to connect your MIDI controller device to Rosegarden’s “external controller” input and output MIDI ports.
MIDI control change messages can be used to adjust mixer track settings based on the incoming MIDI channel from your controller device.
Rosegarden also supports MIDI controller numbers 81 and 82 for changing the currently active window and the selected track in the main window, respectively.
The behavior of MIDI controller number 82 depends on the number of tracks in the window, ensuring it uses the full MIDI controller data range.

6. Special Parameters

The Track Parameter Box assign instruments to tracks control what incoming MIDI data from which sources will be recorded control staff attributes govern how MIDI segments will be created for each track

Playback parameters alternative mechanism to the menu-based system of instrument assignment
Track recording filters control what events are recorded to this track all setting is safe and convenient if you are recording only one MIDI track
Staff export options normal, small, or tiny size
Create segments with means of controlling many of the segment parameters for segments that don’t yet exist aimed especially at notation users

The Segment Parameter Box modify some aspects of the selected segments The Instrument Parameter Box which instrument is assigned to the current selected track

MIDI Instrument Parameters percussion toggle that controls whether only percussion programs should be offered shows you the controllers available for this device
Audio Instrument Parameters two main faders Audio instruments are always ultimately stereo five buttons at lower left are the effects slots.
Synth Plugin Instrument Parameters most important control here is the button at top left

7. Quantization

Quantization is the process of adjusting note and event positions so that they align with exact beat divisions, typically used to achieve precision or clear notation.
In Rosegarden, you can perform general quantization using Segments->Quantize in the main window or Adjust->Quantize in matrix or notation editors.
The quantize dialog in Rosegarden provides various quantization options, including the
- grid quantizer moves the start times of notes so that they start at the nearest point on some grid has adjustable parameters: base grid unit, swing,iterative amount, duration with start times several ways to apply grid quantization, including using the Quantize menu in the Segment Parameter Box and a quantize control in the matrix view.
- legato quantizer carries out a simple grid quantization, and also adjusts the note durations so as to remove any gaps between notes; also available with Adjust->Quantize->Legato function in the Matrix editor
- heuristic notation quantizer adjust note start times and durations to produce readable notation can configure complexity, base grid unit, tuplet level, re-beam, articulation, tie, split-and-tie
Rosegarden provides

Explore the possibilities here thoroughly including raw note ruler !!!

8. The Matrix editor

The Grid and Quantize controls

The Rosegarden matrix editor is used for viewing and editing music in a grid format, with each note represented as a block at a height corresponding to its pitch.
Each matrix editor window displays a single segment of music, and you can access it by right-clicking on a segment in the main canvas and selecting the matrix editor from the menu.
The matrix editor provides Grid and Quantize controls in its main toolbar.
The Grid menu controls the size of the visible grid and the time resolution for note insertion and movement, allowing for snapping to grid points.
The Quantize menu performs quantization, adjusting selected notes or the entire segment to a chosen grid resolution.
The Quantize menu continuously shows the current quantization resolution for the selected segment or notes.

Inserting notes

In the matrix view of Rosegarden, you can enter notes using various methods.
The simplest way is to use the mouse with the pencil tool selected. Click and drag to place a note on the matrix editor at a specific time and pitch.
The Select tool (arrow) also allows you to draw notes using the middle mouse button or the left mouse button with Ctrl pressed.
Note durations are snapped to the grid unit defined by the Grid setting, which you can adjust.
You can enter notes using the PC keyboard by selecting the desired duration with number keys and then pressing pitch keys (A, S, D, etc.) to input notes. Shift sharpens notes, and Shift+Ctrl flattens them.
For notes an octave higher or lower, use the corresponding keys on the keyboard.
All insertion commands are available in the Tools menu, and keyboard shortcuts are designed for a QWERTY-layout keyboard. [use keyboard shortcuts to get dvorak? !!!]
You can also enter notes using a MIDI keyboard or other MIDI control device, with durations determined by the Grid setting, in a process called “step recording.”

Selecting notes

In the matrix editor of Rosegarden, you can select notes using the Select (arrow) tool.
To select notes, outline a rectangular area that contains the notes you want to select. Clicking on a single event selects only that event.
Holding Shift while selecting notes adds the new selection to any existing selection instead of replacing it.
To clear a selection, click in an empty space on the matrix editor or press the Escape key.
You can select all notes of a specific pitch by shift-clicking on a key in the piano keyboard on the left side of the matrix editor. Shift-click and drag to select a range of pitches.
The Edit->Filter Selection function allows you to filter a range of events in various categories out of your selection, either inclusively or exclusively. [This is neat, though I’m not sure what to use it for.]
Once your selection is filtered, you can manipulate it using conventional editing methods.

Moving and copying notes

In the Rosegarden matrix editor, you can move notes in both time and pitch.
To move notes, you can either click and drag them with the Move tool or select them and then drag them using the Select tool.
To copy notes, select them and use standard copy and paste functions. After pasting, the pasted notes will be selected, and you can further drag them to their desired position.
Alternatively, you can copy notes by selecting them with the Select tool and then holding Ctrl while click-and-dragging the selected notes to a new position.

9. The Percussion Matrix editor

The Percussion Matrix editor is similar to the regular Matrix editor but displays pitch names on the vertical scale instead of a piano keyboard.
It is designed for use with percussion programs on MIDI instruments, where each note pitch corresponds to a different percussion sound.
The display of pitch names depends on whether the segment being edited is playing through a percussion instrument with associated key mappings to define the names.
If key mappings are not defined, the standard matrix view piano keyboard will be displayed instead.

10. The Notation editor

Notes To optimize windowq with leftwm:

Put track editor into float H-S-t
Open notation editor as float
Resize notation editor with H-<mouse-right> and it will come to the top

(This system works really well!)

Midi -> rg doesn’t translate well for ly always eg grace notes are written out.

EAN experiment as needed

Introduction

The Rosegarden Notation editor allows users to view and edit one or more staffs in traditional score notation.
It supports various musical elements such as chords, overlapping and nested beamed groups, triplets, tuplets, grace notes, dynamics, accents, and text.
While it provides good support for music notation, it does not offer the same level of fine control over layout as dedicated score editing programs.
Rosegarden is not focused on producing typeset-quality scores but can export LilyPond files that may serve as a basis for typesetting.

Single and multiple staffs

There are three ways to open a notation editor in Rosegarden: double-clicking on a segment, using the “Open in Default Editor” menu function (Return key), or using the “Open in Notation Editor” menu functions.
Double-clicking on a segment opens it in the notation editor, and you can also open multiple selected segments together in a single notation window by holding Shift and double-clicking the last one.
The “Open in Default Editor” menu function works similarly to double-clicking and opens selected segments in the default editor.
The “Open in Notation Editor” menu function allows you to select segments and open them in the notation editor. You can switch between staffs within the notation editor, with the topmost staff being the default current staff.

Linear and page layouts

The default display mode in the notation editor is the Linear layout, where staffs are shown one on top of another in a single horizontally scrollable line. [All functions including various rulers are available here, so this should be where we do most of the work.]
The notation editor also offers two page-based layouts: Continuous Page and Multiple Page.
In Continuous Page layout, music is displayed in one infinitely-tall page with each staff breaking at the end of a line.
In Multiple Page layout, music is divided into real pages based on the specified printing resolution.
Some display and editing functions work in both page layout modes and linear layout.
You can switch between these layout modes using menu functions: Linear Layout, Continuous Page Layout, and Multiple Page Layout.

Rulers and cursors

The Rosegarden Notation editor includes two graduated rulers: one at the top and one at the bottom of the main notation canvas.
There are several types of cursors associated with these rulers, including the insert cursor (purple) and the playback pointer (blue).
The insert cursor marks the point for operations like insert and paste and can be repositioned using various methods, including clicking on the ruler or using arrow keys.
The playback pointer indicates the current playback or record position and can also be repositioned.
The raw note ruler is useful for editing quantized notation, displaying the actual performance duration of notes. [Can probably tweak durations with it. !!!]
The chord name ruler attempts to display the chords that are sounding in the piece but is for informational purposes only. [Can be useful to understand chord progressions. !!!]
The tempo ruler displays the global tempo and allows for entering, editing, and manipulating tempo changes. [Can ramp with this, though doing so may be better in lmms. !!!]
These rulers and cursors are typically shown in Linear layout mode and may not all be displayed by default [can be changed in View].
You can enable or disable certain rulers and cursors through settings and configuration options.

Tools and selections

The Rosegarden Notation editor offers various tools for editing, including selection, erasing, inserting notes, rests, text, and more.
Functions in the editor fall into 3 categories like tools, functions that operate on selections, and drag operations.
Choosing a tool changes the default behavior of the mouse buttons on the notation canvas.
Selections are made using the Select (arrow) tool, which allows you to outline a rectangular area to select events. Selection are only from one staff and can be filtered.
Shift will accumulate selection contents. Double-click selects bar. Triple-click selects segment.
Many menu functions require a selection to be made on the canvas before they can be used, and the Edit menu offers options for selecting extended areas.
Dragging can also be used for editing, such as changing pitch by dragging note heads up or down or changing the timing of notes by dragging them left or right.
There are functions for transposing notes by a specified interval, including transposing by semitones or intervals with a dialog for customization.
Double-clicking on a single event opens an editing dialog for that event.

Inserting notes and rests

To insert a note in the Rosegarden Notation editor, select a note duration from the toolbar and then click on the staff at the desired point.
Accidentals can be selected from the toolbar to insert sharpened or flattened notes, or you can use the “Follow previous accidental” icon to follow the last accidental used at the same height on the staff.
Inserting a note at the same time as an existing note of different duration will create chords and possibly ties.
Adding a note exactly over an existing note of different duration will change the duration of the existing note.
Adding a rest at the same time as an existing note will silence the note for the duration of the rest.
Typing notes and rests with the PC keyboard involves selecting a duration and pitch using number and letter keys. You can also toggle between dotted and undotted durations [with . and triplet version of note with G].
Rests can be inserted by pressing “P” [pressing P will change notes into equivalent rests], and you can switch between note and rest tools using “T”.
Chord mode can be activated using the “H” key to insert subsequent notes at the same time as the last one. [This is quite effective if entering notes from a computer keyboard.]
Keyboard shortcuts for changing note durations are available, and holding Ctrl while pressing a number key selects the corresponding dotted-note tool.[Doesn’t work !!!]
Step recording with a MIDI keyboard is possible by configuring the keyboard as the default MIDI record device, selecting a duration, and using the step recording button on the toolbar or the Tools menu.
Note durations can be changed using keyboard shortcuts, and you can enter notes of different durations during step recording.

Be aware of some of these shortcuts over time because they can simplify entry. !!!

Clef and key

In Rosegarden, time-related changes like tempo and time signatures apply uniformly to all staffs, ensuring consistent playback and notation.
[You can change tempos, and time signatures in the Composition dropdown.]
[You can change clefs, keys, in the Segment dropdown.]
When changing the clef, you can choose to maintain current pitches or transpose into an appropriate octave for existing notes following the clef change.
When changing the key, you can select the key signature and decide whether to transpose it according to segment transposition or use it as specified.
The scope of the key change can be applied to the current segment only or to all segments on all tracks at that time.
You can specify how existing notes following a key change should behave, whether maintaining their current pitches, maintaining current accidentals, or attempting to transpose into the new key.

Some of these may require trying out as necessary. !!!

Text, Lyrics, and LilyPond directives

Rosegarden allows you to add various types of text to a staff, including single text items for annotations, lyrics, and LilyPond directives.
The Text tool is used for creating single text items and lyrics. You can click at the point where you want to add text, and a dialog box will appear for you to enter the text and choose its intended purpose.
There are different styles or purposes for text, such as Dynamic, Direction, Tempo, Chord, and more. These styles determine how the text is displayed. [Some annotations may be turned off by default.]
Lyrics can be edited using the Lyric editor, which follows specific formatting rules with bar lines and syllables. [Do not do lyrics in rg - go to lilypond instead.]
LilyPond directives are special text used for exporting .ly files with extended functionality not available through Rosegarden’s native interface.
LilyPond directives include Segno, Coda, Alt1, Alt2, double barlines, terminating barlines, dashed barlines, glissando, arpeggio symbols, and font size adjustments like Tiny, Small, and Normal.
Proper placement of LilyPond directives within a measure can affect their function and appearance in the exported score. [Some experimentation is required especially with the Lilypond directives which can likely be very helpful if they translate over properly in to the text - they really don’t look particularly good in the notation editor though because they appear as text. However, clicking on the note using T gives choice of directive and that looks just fine in the lp export!]

Guitar Chords

You can add guitar chord diagrams to a notation track in Rosegarden.
To add a guitar chord diagram, toggle the Guitar Chord button and then click on the area where you want to add it.
Rosegarden provides a chord dictionary with common chord fingerings categorized as beginner, common, and all.
You can also create your own chord fingerings by clicking the ‘New’ button, which opens a chord editor where you can draw the fingering, specify the root, chord extension, and starting fret number.
When adding your own fingerings, you can type custom extension values, but they will only be available in the ‘all’ list.
You can delete and edit your custom chords, but predefined dictionary chords cannot be changed, and any mistakes should be reported to the development team.

This is actually pretty neat stuff because rg figures out the chord for you, though it can be changed.

Slurs and hairpins

Slurs and hairpins in Rosegarden are stored as events, similar to notes or rests.
To create a slur or hairpin, select the area (a series of notes or other elements) you want it to cover, then use the “Add Slur,” “Add Crescendo,” or “Add Decrescendo” options from the Phrase menu.
The placement of hairpins can be challenging, especially when they run in line with dynamic markings. Sometimes, their appearance in Rosegarden’s notation editor may differ from the final result when exported to LilyPond. [There seems to be no cresc or dimin text as lp offers. Hairpins can be moved sideways but not up|down.]
Rosegarden’s slurs work best when all the stems of the notes are running in the same direction, and they may need manual repositioning for better placement. Slurs tend to improve their appearance after exporting to LilyPond.
You can remove a slur or hairpin by selecting it with the selection tool and hitting Delete or by clicking on it with the erase tool active.

Note stems and slashes

In Rosegarden’s Notation editor, you can adjust the stem direction of selected notes using the “Stem Direction” options found in the Notes menu.
These options allow you to set the stems either up or down, or to revert to the stem direction that Rosegarden would choose automatically.
Additionally, the Note menu offers options for adding slashes across note stems, which are used for notational purposes, such as indicating short repeated notes or drum rolls. These slashes do not affect playback or MIDI generation in Rosegarden.

Beams

Beams in the Rosegarden Notation editor are used for grouping notes together.
To add beams to selected notes, use the “Beam Group” option found in the Phrase menu.
To remove a beam from selected notes, use the “Unbeam” menu option.
You can have multiple beamed groups simultaneously, but you may need to manually set stem directions for each group.
Rosegarden can automatically guess sensible beaming for a piece of music; use the “Auto-Beam” option on the Phrase menu to invoke this feature.
Automatic beaming is also done when loading a MIDI file, and during note entry (which can be disabled in settings).

Triplets and other tuplets

A triplet group consists of three notes or chords played in the time of two.
The term “tuplet” refers to a situation where notes with a total duration of any given number of units are played in the time of a different number of units (e.g., nine notes in the time of eight, six in the time of four).
Rosegarden’s triplet and tuplet menu functions squash existing notes or rests to play quicker, filling in the leftover space with a final rest and representing the whole as a tuplet group.
You can create triplet groups by entering the first two notes, making them into triplets, and then filling in the rest with the final note. Alternatively, you can apply the triplet operation before entering any notes, effectively turning the rest into triplet form and inserting the notes on top of it. [Good way is to put the 3 notes down, select them, then triplet with C-r; better way may be to go into triplet mode with g|G, and just enter the notes.]
To specify a general tuplet group, you need to know the base note (e.g., a crotchet) and the ratio of the number of base notes written to the number played. [Concept should be similar to triplets, but setup isn’t automatic.]

EAN!!!

Paste types

Rosegarden’s Notation editor provides various types of Paste operations to handle music editing, as traditional cut-and-paste is less straightforward in music.
The available Paste types include Restricted, Simple, Open and Paste, Note Overlay, and Matrix Overlay.
Restricted Paste requires an existing gap of sufficient duration to contain the clipboard’s contents, while Simple Paste erases events to create a gap for pasting.
Open and Paste makes room for the paste by moving subsequent events, while Note Overlay and Matrix Overlay handle pasting notes in specific ways with respect to overlaps and ties.
These paste types can be accessed through the Paste… menu option on the Edit menu.

Matrix overlay is unclear - seems to be the same as Simple !!!

Adjusting Notation

The “Normalize Rests” function in Rosegarden’s Notation editor can be used to adjust, split, and merge rests to ensure they have correct durations and fall on appropriate boundaries within the time signature. [Eg q h q in 4/4 -> w]
“Tie Notes at Barlines” is a function designed to split excessively long notes into shorter tied notes, particularly when they overflow barlines or cannot be displayed as a single note.
“Split-and-Tie Overlapping Chords” is a function used to split and tie overlapping notes and chords, helping to create neat blocks of chords and single notes. [!!!]
“Halve Durations” and “Double Durations” functions allow for simple rescaling of note durations, while more complex rescaling can be achieved using the “Stretch or Squash” function. [!!!]
To reposition notation elements like slurs, hairpins, and text events, you can use Ctrl-click and drag. However, this method cannot be used to reposition notes. [Works but not with much control.]

These are good ways to make the notation neater and thus better formed for lilypond probably. !!!

Interpreting Performance Cues in Notation

The “Interpret” function in Rosegarden’s Notation editor allows you to modify note velocities and timings based on indicated dynamics.
Interpretations available include applying text dynamics (e.g., p, mf, ff), making notes gradually increase or decrease in velocity during crescendo or decrescendo hairpins, stressing beats, and articulating slurs, staccato, and tenuto markings.
Text dynamics such as pp, p, mp, mf, f, ff, and fff are matched to set velocities based on the last “Dynamic” text seen on the same staff.
Crescendo and decrescendo hairpins adjust note velocities to create a gradual increase or decrease in loudness.
Stressing beats makes notes landing on bar or beat boundaries slightly louder than surrounding notes.
Articulating markings like slurs, staccato, and tenuto affect note lengths, with unslurred notes shortened, staccato notes further shortened, and notes inside slurs and tenuto markings played at their full length.

[Couldn’t get this to really work - not important though for notation. !!!]

Notation from performance data

Rosegarden, while primarily a sequencer, offers notation editing capabilities, which require adjusting note times and durations for musical precision.
This quantization, known as “notation quantization,” improves the accuracy of notes shown in the notation editor.
Notation quantization affects only what is visible in notation and doesn’t impact notes played or edited in other views.
You can see the effects of notation quantization in the notation editor through the raw note ruler or by highlighting altered note heads in green.
Rosegarden provides various quantization parameters that allow you to customize the quantization process, including complexity, base grid unit, and tuplet level.
You can also control how the quantizer tidies up after quantization, including re-beaming notes, adding articulations like staccato, tenuto, and slurs, tying notes at barlines, and splitting and tying overlapping chords.
These quantization settings can be adjusted to suit different musical contexts and preferences.

Again more to experiment with in the quantization section. !!!

Playing ornaments

Rosegarden supports playback of ornaments using triggered segments.
To create a new ornament, write out the desired notes, select them, and use the “Note->Ornaments->Make Ornament…” option. This creates a triggered segment.
To make a note play an existing ornament, select it and use “Note->Ornaments->Trigger Ornament…”
You can edit the notes played in a triggered ornament by double-clicking on the triggering note.
Note that Rosegarden can adjust the pitch of triggered segments to match their triggering notes but cannot transpose into a different key or scale position.

Figure out how to setup a file with ornaments defined - didn’t get an answer to this from the rg list, but the article Ornaments: Pleasure and Pain may have a solution. !!!

Note styles

Rosegarden allows customization of how different types of notes are visually represented, including note heads, tails, and fill options.
Standard styles like Classical, Cross, Triangle, and Mensural are provided for note appearance.
You can change the style of individual notes by selecting them and using the Note Style menu options.
Each note remembers its chosen style, and this information is saved as part of the composition.
You can also set a default style for new notes using the “Configure Rosegarden…” option in the Settings menu.
Advanced users can create custom note styles by describing them in XML files. The standard styles are also defined in this way.

Note fonts

Rosegarden comes with a default scalable notation font called Feta, developed by the LilyPond project, suitable for classical music. [It seems to be all that is required.]
Rosegarden supports additional notation fonts, but you need to provide them in TrueType (.ttf) or PostScript Type-1 (.pfb, .pfa) format and have the necessary permissions to install them in the Rosegarden directory.
To install a supported notation font, copy the font files into the “fonts” subdirectory of the Rosegarden installation directory and restart Rosegarden.
Supported fonts include Fughetta, Petrucci, Maestro, Opus, Inkpen, Sonata, Steinberg, and Xinfonia, but it’s your responsibility to ensure you have the legal rights to use them.
You can also create new mapping files to use new fonts, especially if they have similar mappings or metrics to existing fonts.
For more details on customizing fonts and mappings, refer to the “Customising Rosegarden” section.

11. The Event List editor

The Event List editor in Rosegarden displays events within a single segment in a raw form.
It allows for precise editing of individual event properties.
This editor provides a detailed view of events within a segment for fine-tuning and customization.
You can access the Event List editor in Rosegarden for working with events in your compositions.

Probably is merit in learning this for the purpose of fine-tuning. !!!

12. Customising Rosegarden

Configuring custom notation fonts

Customizing Rosegarden involves configuring custom notation fonts.
Unsupported notation fonts in TrueType or Type-1 formats can be supported through mapping XML files.
Mapping XML files describe font character maps and metrics for Rosegarden’s use.
Custom fonts can be pixmap-based or scalable system fonts.
Mapping files include elements like font information, origin, copyright, type, and more.
Custom fonts can be antialiased (smooth) if specified.
Scalable fonts can have autocrop attributes to adjust spacing.
Font requirements are used for scalable fonts and allow font composition from multiple system fonts.
Font-sizes element specifies available notation font sizes and their relationships to non-font elements.
Font-scale and font-size elements define font proportions and sizes.
Font-symbol-map lists symbols in the notation font and their sources, including pixmap files or system font code points.
Various attributes like name, src, code, glyph, and more are used to describe symbols.
Customization allows users to tailor notation fonts to their preferences.
Mapping XML files are typically stored in the fonts/mappings subdirectory of the Rosegarden installation directory.
Users can edit provided mapping files or create their own to customize fonts effectively.

Creating new notation styles

Rosegarden’s notation editor can display and edit notes in various standard styles.
Notation styles are defined in XML style definition files.
Users can create new notation styles by writing simple XML files.
Rosegarden determines which styles to offer based on installed style files.
Default style examples can be found in the styles subdirectory of the Rosegarden installation directory.
The XML format for notation styles includes elements like “rosegarden-note-style,” “global,” and “note.”
Attributes within these elements define parameters for note styles, such as note type, shape, font character name, filled head, stem presence, flags, slashes, and more.
Users can base new styles on existing ones by specifying a “base-style.”
Feedback on creating new styles is welcomed on the Rosegarden mailing-lists.
The file name, not the “rosegarden-note-style” element, currently defines the style name, but this may change in the future.
The XML format is designed to describe standard styles that come with Rosegarden and may not support extensive customization.
Users can define note head shapes and fonts for different note styles.
Styles can specify the presence of stems, flags, slashes, and their positions relative to note heads.
Rosegarden’s notation editor provides flexibility in creating and customizing notation styles.

13. Credits and License

14. Rosegarden Revision History

Last revision seems to be May 2008

zz

NOTE4: we can use rg for input (-> emacs lilypond ) and playback (better than qtractor|lmms ?) NOTE3: why we should not use rg for notation

it is very awkward
it does not produce highest quality lilypond
we aren’t using quantization anyway (which doesn’t work too well either)

NOTE2: the export was not the problem - the template time signature was! NOTE1: after considerable experimentation, it seems the ly export is not of high quality and is filled with errors. therefore we need to consider alternative system. additionally there are continuous crashes when using timidity (though it was good with yoshimi).

slurs do not sync with lyrics automatically, so use . to make lyric go for more than a single note.

MIDI/audio sequencer and notation editor (not exactly a daw) (far better playback control than frescobaldi!)

https://wiki.archlinux.org/title/Rosegarden handbook make sure fluidsynth has been setup or rebooting is necessary for rosegarden to see there is a midi

rethink the usage of rg because in the past we just wanted it as a pre-lilypond interface however, since it can do much more a proper examination is required plan

dakloos midi imported using default yoshimi arpeggio sounds really neat!

Lilypond

Sun, 19 May 2024 23:57:00 -0700

LilyPond is a free program that lets you create beautiful sheet music for all kinds of music, from classical to modern, just like for piano or guitar.

The coding commands will have emacs shortcuts attached to them as these are produced.

fundamental concepts

file structure

for instance,

Every block is a separate chunk of music within a block. multiple scores -> still one output file

Every block affects the or block in which it appears multiple books -> separate output file

note structure

pitch duration post-event(s)

brackets and ties

{ … } Encloses a sequential segment of music < … > Encloses the notes of a chord << … >> Encloses simultaneous music expressions eg voices ( … ) Marks the start and end of a slur $…$ Marks the start and end of a phrasing slur [ … ] Marks the start and end of a manual beam

x/y { … }

voices

<< d
>>

Staff << Voice = “1” { } Voice = “2” { } >>

musescore transfer

musescore -> xml musescore $f -o ${f%.*}.musicxml

xml -> ly musicxml2ly –nrp –npb –npm –npl –nsd –nb -m -r -l english $f

automatic scaling to paper size

#(set-default-paper-size “letter”) at top level will automatically scale and it is not necessary to do it with then diddle in here (as originally done): Score.NonMusicalPaperColumn.padding = #1.3

emacs

textedit

follow instructions for gnome3 in 4.1.1 Configuring the system

lilypond-invoke-editor.desktop already in /usr/share/applications [Desktop Entry] Version=1.0 Name=lilypond-invoke-editor GenericName=Textedit URI handler Comment=URI handler for textedit: Exec=lilypond-invoke-editor %u Terminal=false Type=Application MimeType=x-scheme-handler/textedit; Categories=Editor NoDisplay=true

then execute: xdg-desktop-menu install /usr/share/applications/lilypond-invoke-editor.desktop xdg-mime default lilypond-invoke-editor.desktop x-scheme-handler/textedit

put in .zshenv: export LYEDITOR=emacsclient the scm/editor.scm already has emacsclient in it so no special config necessary

scheme

Scheme tutorial
1.1 Introduction to Scheme
1.1.1 Scheme sandbox
1.1.2 Scheme variables
1.1.3 Scheme simple data types
1.1.4 Scheme compound data types
1.1.5 Calculations in Scheme
1.1.6 Scheme procedures
1.1.7 Scheme conditionals
1.2 Scheme in LilyPond
1.2.1 LilyPond Scheme syntax
1.2.2 LilyPond variables
1.2.3 Debugging Scheme code
1.2.4 Input variables and Scheme
1.2.5 Importing Scheme in LilyPond
1.2.6 Object properties
1.2.7 LilyPond compound variables
1.2.8 Internal music representation
1.3 Building complicated functions
1.3.1 Displaying music expressions
1.3.2 Music properties
1.3.3 Doubling a note with slurs (example)
1.3.4 Adding articulation to notes (example)
Interfaces for programmers
2.1 LilyPond code blocks
2.2 Scheme functions
2.2.1 Scheme function definitions
2.2.2 Scheme function usage
2.2.3 Void scheme functions
2.3 Music functions
2.3.1 Music function definitions
2.3.2 Music function usage
2.3.3 Simple substitution functions
2.3.4 Intermediate substitution functions
2.3.5 Mathematics in functions
2.3.6 Functions without arguments
2.3.7 Void music functions
2.4 Event functions
2.5 Markup functions
2.5.1 Markup construction in Scheme
2.5.2 How markups work internally
2.5.3 New markup command definition
2.5.4 New markup list command definition
2.6 Contexts for programmers
2.6.1 Context evaluation
2.6.2 Running a function on all layout objects
2.7 Callback functions
2.8 Unpure-pure containers
2.9 Difficult tweaks

Netricks

Sat, 05 Aug 2023 11:53:00 -0700

Here are some useful things that can enhance your net efforts with websites, email, aibots, and more!

mailto

The mailto directive can greatly facilitate the emailing process and is usually used on contact forms on a webpage. The technique can also be used in actual emails that html formatted.

The mailto is quite simple to setup using basic html:

<a href="mailto:username@domain-name.top-level-domain">

username might be something like prad.e.ception
domain-name might be githubris
top-level-domain could be items like com, org, info, etc

Below we provide several examples demonstrating various situations involving mailto.

No mailto

Let’s start with what most people do because they do not know any html whatsoever. They will put something like this in an email or webpage:

Hey! Email me here: prad.e.ceptor@githubris.info

Well that’s fine, but if you want to email prad, you have to copy and paste that email address into your email client (eg gmail, outlook, evolution, mutt, notmuch or whatever you use). Such an effort could be too much for some and poor prad would never hear from them!

Simplest mailto

So to make things more efficient we can enclose the email inside a mailto code like this:

Hey! Email me <a href="mailto:prad.e.ceptor@githubris.info">here</a>!!

The above code produces the following clickable link:
Hey! Email me !!

You can just click on the here link and the email will be produced by your email client all set to go! Give it a try, but don’t send it because that email address isn’t a real one.

Throw in a subject

Suppose you want people to contact you for a specific reason. Rather than having them fill in the subject field of the email, you can do it for them, like this:
#+begin_src html Hey! Email me <a href="mailto:prad.e.ceptor@githubris.info?subject=Tell me about physics">here</a> about physics!!

You will get the following clickable link:
Hey! Email me [here](mailto:prad.e.ceptor@githubris.info?subject=Tell me about physics!) about physics!

Ya the here link looks the same, but it isn’t! Try it and see what happens!

Email more than one recipient

Let’s send that email to not just prad, but also copy (cc:) it to brad@copycater.com, and blind copy (bcc:) it to clad@cheapoutfittings.org (understand that none of the emails exist, so if you are a spammer, you’ll be disappointed!):

Hey! Email us all <a href="mailto:prad.e.ceptor@githubris.info?subject=Tell me about physics!&cc=brad@copycater.com&bcc=clad@cheapoutfittings.org">here</a>, ok!!

You get to click the same looking here link, but it’s not the same:
Hey! Email us all [here](mailto:prad.e.ceptor@githubris.info?subject=Tell me about physics!&cc=brad@copycater.com&bcc=clad@cheapoutfittings.org), ok!!

You can keep adding to the cc: (or bcc:) fields by separating them with the rather cryptic %2C%20 (where %2C is really a comma, and %20 is really a space). A simple comma may work, but this sort of stuff can get messy!

It doesn’t have to though, because you can use mailto creating tools such as this mailto-link-generator which will make the job easy!

A real opportunity!

Let’s use the mailto-link-generator for a worthy cause!

The dynamic group Animal Justice is working to hold companies that sell eggs from caged hens accountable for such cruelty! One of these companies, The Second Cup, had promised to go cage-free, but is apparently reneging on that commitment.

Let’s use the mailto-link-generator and send an email to The Second Cup departments, specifically:

General Consumer Inquiries - rewards@secondcup.com
Marketing - marketing@secondcup.com
Franchising - franchising@secondcup.com

The subject will be

Go Cage Free!!

The message (you can indeed include the body text!) will be somewhat terse for there is no reason to be excessively nice to exploitative oppressors:

What happened to your promise to animal welfare! Your website stated that you were ensuring your products would be made with 100% cage-free eggs.

Now, that sentence has been deleted from the website!

It is still stated on your website that The Second Cup is committed to animal welfare and to ensuring that your suppliers’ policies and practices align with your values.

So which is it? Are you going cage-free or not?!

As a consumer, I want to know whether I will spend my money at a seller who keeps promises, or relegate to the cellar, one who breaks them!

Are you ready?

Go to the mailto-link-generator and start filling in the fields:

To field: rewards@secondcup.com
Cc field: marketing@secondcup.com, franchising@secondcup.com
Subject field: Go Cage Free!!
Message field: copy and paste the above message or type up your own. (Sign it with your name.)
Link text field: Email The Second Cup

Then press the Generate button and you’ll see your mailto link and even the entire html code which you can insert into an html email message or a webpage respectively. You can click on the Copy buttons to grab the code and use it like this:

[Email The Second Cup](mailto:rewards@secondcup.com?cc=marketing@secondcup.com, franchising@secondcup.com&subject=Go%20Cage%20Free!!&body=What%20happened%20to%20your%20commitment%20to%20animal%20welfare!%20Your%20website%20stated%20that%20you%20were%20ensuring%20your%20products%20would%20be%20made%20with%20100%25%20cage-free%20eggs.%0A%0ANow%2C%20that%20sentence%20has%20been%20deleted%20from%20the%20website!%0A%0AIt%20is%20still%20stated%20on%20your%20website%20that%20Second%20Cup%20is%20committed%20to%20animal%20welfare%20and%20to%20ensuring%20that%20your%20suppliers’%20policies%20and%20practices%20align%20with%20your%20values.%0A%0ASo%20which%20is%20it%3F%20Are%20you%20going%20cage-free%20or%20not%3F!%0A%0AAs%20a%20consumer%2C%20I%20want%20to%20know%20whether%20I%20will%20spend%20my%20money%20at%20a%20seller%20who%20keeps%20promises%20or%20should%20be%20relegated%20to%20the%20cellar%20for%20breaking%20them!)

Click on the above link and see what your email client prepares for you! Then send it to The Second Cup, so they have no doubt that we are watching them!

Want to know more?

You now have enough mailto knowledge to be dangerous!

If you want to go further into this, try these links:

Or if you are a I-don’t-read-the-book-so-show-me-the-movie type, watch away:

Imagemagick

Tue, 21 Jun 2022 20:07:52 -0700

Since we are going text/prg based for most things, we may consider imagemagick as a fairly regular replacement for gimp. This effort will also lead to understanding graphic software terminology.

Imagemagick website and Examples of ImageMagick Usage

Waiting items below will be elaborated on and some eliminated.

Quick Fixes

stitching gifs together

as explained here, the idea is to separate frames, join them, put it all back together: convert 1.gif -coalesce a-%04d.gif convert 2.gif -coalesce b-%04d.gif

for f in a-*.gif; do convert $f ${f/a/b} +append $f; done # append frames

convert -loop 0 -delay 20 a-*.gif result.gif # rejoin frames

montage

in a dir with jpg,png this is the sort of thing that can be done: magick montage -geometry 99x55>+2+2 -tile 4x1 *.{jpg,png} 00.jpg

crop

magick amazon-homepage.png -crop 2560x1440+1920+0 tmp.png

can use magick display to determine parameters though it is a bit awkward

make png transparent

convert test.png -transparent white trans.png

Features and Capabilities

Command-line processing: utilize ImageMagick from the command-line.

ImageMagick command-line consists of

one or more required input filenames:
- filename globbing magick *.jpg images.gif
- explicit image format magick -size 640x480 -depth 8 rgb:image image.png
- using built-in images and patterns magick -size 640x480 pattern:checkerboard checkerboard.png
- STDIN, STDOUT, and file descriptors not sure what is going on??
- selecting certain frames from an image magick ‘images.gif[0]’ image.png magick ‘images.gif[0-3]’ images.mng
- selecting a region of an image magick -size 6000x4000 -depth 8 ‘rgb:image[600x400+1900+2900]’ image.jpg
- forcing an inline image resize magick ‘*.jpg’ -resize 120x120 thumbnail%03d.png but better magick ‘*.jpg[120x120]’ thumbnail%03d.png
- forcing an inline image crop magick ‘*.jpg’ -crop 120x120+10+5 thumbnail%03d.png
- filename references magick @myimages.txt mymovie.gif magick image-%d.jpg[1-5]
zero, one, or more image settings.
zero, one, or more image operators.
zero, one, or more image sequence operators.
zero, one, or more image stacks.
zero or one output image filenames

Resizing an image

magick tmp.jpg -resample 10% -sharpen 0x2 -background none -flatten 00.jpg

Without the last 3 parameters the corners came out black instead of transparent.

Change black <-> white

magick input.png -channel RGB -negate output.png complements every color.

To just to the blacks and whites PP says use:

magick input.png \
  \( -clone 0 -colorspace HSL -channel Lightness -separate +channel -negate \) \
  \( -clone 0 -fill white -colorize 100 \) \
  -compose Luminize -composite \
  output.png

This command does the following:

Creates a copy of the input image and extracts its lightness channel.
Negates the lightness channel, which inverts black and white.
Creates another copy of the input image and turns it completely white.
Combines the negated lightness with the white image using the Luminize composition method.
This approach will invert black and white while preserving other colors14. It works by manipulating the lightness channel in the HSL colorspace, which affects only the brightness of the pixels without changing their hue or saturation6.

For even more control:

magick input.png \
  \( -clone 0 -colorspace HSL -channel Lightness -separate +channel -threshold 5%,95% -negate \) \
  \( -clone 0 -fill white -colorize 100 \) \
  -compose Luminize -composite \
  output.png

Waiting

Animation: create a GIF animation sequence from a group of images.

Bilateral blur: non-linear, edge-preserving, and noise-reducing smoothing filter.

Color management: accurate color management with color profiles or in lieu of– built-in gamma compression or expansion as demanded by the colorspace.

Color thresholding: force all pixels in the color range to white otherwise black.

Complex text layout: bidirectional text support and shaping.

Composite: overlap one image over another.

Connected component labeling: uniquely label connected regions in an image.

Convex hull: smallest area convex polygon containing the image foreground objects. In addition, the minimum bounding box and unrotate angle are also generated.

Decorate: add a border or frame to an image.

Delineate image features: Canny edge detection, Hough lines.

Discrete Fourier transform: implements the forward and inverse DFT.

Distributed pixel cache: offload intermediate pixel storage to one or more remote servers.

Draw: add shapes or text to an image.

Encipher or decipher an image: convert ordinary images into unintelligible gibberish and back again.

Format conversion: convert an image from one format to another (e.g. PNG to JPEG).

Generalized pixel distortion: correct for, or induce image distortions including perspective.

Heterogeneous distributed processing: certain algorithms are OpenCL-enabled to take advantage of speed-ups offered by executing in concert across heterogeneous platforms consisting of CPUs, GPUs, and other processors.

High dynamic-range images: accurately represent the wide range of intensity levels found in real scenes ranging from the brightest direct sunlight to the deepest darkest shadows.

Histogram equalization: use adaptive histogram equalization to improve contrast in images.

Image cache: secure methods and tools to cache images, image sequences, video, audio or metadata in a local folder..

Image calculator: apply a mathematical expression to an image, image sequence, or image channels.

Image gradients: create a gradual blend of two colors whose shape is horizontal, vertical, circular, or elliptical.

Image identification: describe the format and attributes of an image.

ImageMagick on the iPhone: convert, edit, or compose images on your iOS device such as the iPhone or iPad.

Large image support: read, process, or write mega-, giga-, or tera-pixel image sizes.

Montage: juxtapose image thumbnails on an image canvas.

Morphology of shapes: extract features, describe shapes, and recognize patterns in images.

Motion picture support: read and write the common image formats used in digital film work.

Multispectral imagery: support multispectral imagery up to 64 bands.

Noise and color reduction: Kuwahara Filter, mean-shift.

Perceptual hash: map visually identical images to the same or similar hash– useful in image retrieval, authentication, indexing, or copy detection as well as digital watermarking.

Special effects: blur, sharpen, threshold, or tint an image.

Text & comments: insert descriptive or artistic text in an image.

Threads of execution support: ImageMagick is thread safe and most internal algorithms execute in parallel to take advantage of speed-ups offered by multicore processor chips.

Transform: resize, rotate, deskew, crop, flip or trim an image.

Transparency: render portions of an image invisible.

Virtual pixel support: convenient access to pixels outside the image boundaries.

Clojure

Thu, 16 Jun 2022 12:54:45 -0700

Clojure is a dynamic, general-purpose programming language, combining the approachability and interactive development of a scripting language with an efficient and robust infrastructure for multithreaded programming.

keybindings

Most are found in cider key ref and should be figured out gradually.

;; docs
C-h . Gives from lsp in eldoc buffer with examples (auto)
(C-u) C-c C-d C-d|w At point (prompt) docs local|web
C-c C-d C-a Apropos for function/var
C-c C-d C-f Apropos for text in documentation
C-c C-d C-e Apropos for text and select !!!
M-. Gives actual code

;; evaluate
C-c C-b Exit evaluation if stuck
C-c C-c Entire current form
C-c C-v C-l Immediate form
C-c|x C-e Previous form
C-c C-k Buffer
C-c M-; Output as comment
C-c M-e Output to REPL
(C-u) C-c C-(M-)l File(s)
C-c C-v (C-)l List
C-c C-v r Region
C-c C-v n The ns form
C-c C-v w Replace form with evaluation
C-c C-v z Preceding top form up to point !!!

;; help
C-h f cider-mode RET

;; launch
C-c M-j Do cider-jack-in-clj
C-c C-x C-x Prompt for choice of REPL

;; macros !!!
C-c C-m Macroexpand !!!

;; move
M-, Return to pre-jump location
C-M-d Down into list
C-M-u Up out of list
C-M-n Forward to next list
C-M-p Backward to prev list
C-M-h Next defun
C-M-@ Next sexp
C-M-f Forward by sexp
C-M-b Backward by sexp
C-M-a Descend into sexp

;; pprint
C-c C-v C-f e|d Pretty print last|current form to popup buffer

;; search
C-c C-d (C-)a Apropos var
C-c C-d (C-)f Apropos doc

;; switch
C-c C-z To relevant REPL
C-c M-z To relevant REPL after eval current buffer
C-c M-n M-n REPL ns to ns of current buffer

;; zzmisc
M-TAB Complete symbol at point
C-c M-d Display REPL connection info
C-c C-o Clear REPL output
C-c M-i Inspect and act on expression
(C-u) C-c C-z REPL jump
C-c M-p Insert form into REPL
C-c C-q Quit the REPL

cider manual

Home

Overview

Programmers are expected to program in a very dynamic and incremental manner, constantly re-evaluating existing Clojure definitions and adding new ones to their running applications.

Features

We need to understand these better to make full use of CIDER.

Interactive code evaluation can do this right in the code with C-c M-; often more useful than repl
Powerful REPL
Code completion
Code reloading
Definition & documentation lookup particularly excellent with C-c d d
Enhanced syntax highlighting and indentation
Enhanced error reporting
clojure.test integration
clojure.spec integration
Interactive debugger
Data Inspector
Integration with Java logging frameworks
Profiling & tracing
ClojureScript support

CIDER in Action

Videos are good for learning CIDER.

What’s Next?

Get familiar with interactive programming and cider-mode

Getting started

Installation

Already done.

Up and Running

Most of this is already setup, but we should look at:

Jacking-in without a Project just clj to get the repl or can also use in a dir M-x cider-jack-in-universal
Connect to a Running nREPL Server after getting a repl running M-x cider-connect for a port

nREPL Middleware Setup

You can skip this section if you don’t plan to use cider-connect or don’t care about the advanced functionality that requires cider-nrepl. However, with tools.deps we can add to ~/.clojure/deps.edn

:cider-clj {:extra-deps {cider/cider-nrepl {:mvn/version "0.49.0"}}
            :main-opts ["-m" "nrepl.cmdline" "--middleware" "[cider.nrepl/cider-middleware]"]}

:cider-cljs {:extra-deps {org.clojure/clojurescript {:mvn/version "1.10.339"}
                          cider/cider-nrepl {:mvn/version "0.49.0"}
                          cider/piggieback {:mvn/version "0.5.3"}}
             :main-opts ["-m" "nrepl.cmdline" "--middleware"
                         "[cider.nrepl/cider-middleware,cider.piggieback/wrap-cljs-repl]"]}

And can then run M-x cider-connect to the repl.

ClojureScript Setup

Overview
Up and Running
Configuration
Using Figwheel
Using shadow-cljs

Using Other REPLs

Alternative Platforms

Overview
Babashka
Nbb
Other Platforms

Using CIDER

Interactive Programming

Overview Instead of repeatedly editing, compiling, and restarting an application, the programmer starts the application once and then adds and updates individual Clojure definitions as the program continues to run … some Clojure programmers have been known to keep a CIDER session running for weeks or even months!
Implementation
Demos
Deep Dive into CIDER

Emacs & Clojure, A Lispy Love Affair

Using cider-mode

C-h f cider-mode is useful, but even better is using which-key-mode to see all key possibilities.

Code Evaluation

Basic: buffer (do first)
Exotic: should be accessed via CIDER’s evaluation commands keymap, prefix C-c C-v
Minibuffer: cider-read-and-eval C-c M-:
Evaluation hooks: ??
Synchronous vs Asynchronous: nrn
Configuration: nrn
Keybindings: C-M-x, C-c C-c, C-c C-v [C-]d

Code Completion

Standard: TAB or M-TAB (setq tab-always-indent ‘complete)
Completion styles: user may want to use a different completion style just for the CIDER complete eg (add-to-list ‘completion-category-overrides ‘(cider (styles basic)))
Auto-completion: with company or corfu (latter already setup via codeium) - not sure if codeium conflicts with corfu or cider when doing its suggestions or incorporates those completions into the suggestions, but it is really not too important.

Code Reloading

Clean everything up and reload one or more namespaces cider-ns-refresh, cider-ns-reload or cider-ns-reload-all

Pretty Printing

Probably NA.

Dealing with Errors

Most of the time CIDER will display the errors in a dedicated buffer … At times, the error being displayed will originate from a bug in CIDER itself. Error displays can be configured.

CIDER presents stack traces in a special major mode, cider-stacktrace-mode. There is a lot that can be done here, so it is worth knowing how to use.

Working with Documentation

C-h . should be initial ref which provides the clojuredocs examples. However, C-c C-d C-d|w can also be useful because the window is easy to close and the latter goes right to clojuredocs. If you want the javadoc use C-c C-d C-j to see it in browser.

Search in docstrings of all loaded variables with C-c C-d C-e|f which can be quite useful if you know only some words in a doc, but don’t really know what the item is.

Cross referencing other documentation can be done inside `` or [[]] in same namespace or outside (fqdn required). Useful if you’d like to be able to point other programmers at different definitions.

Managing Connections

Misc Features

Using the REPL

Basic Usage

Keybindings

Configuration

REPL History

Testing

Running Tests

Test Reports

Supported Libraries

Debugging

Debugger

Enlighten

Inspector

Logging

Macroexpansion

Profiling

Tracing

Configuration

Indentation Specification

Caveats

Troubleshooting

FAQ

Additional Packages

Additional Resources

basics

install: M-x package-install <RET> cider <RET>
run: launch an nREPL or connect to an existing one
- C-c C-x C-x gives menu of REPL choices (view in messsage buffer)
- C-c M-j cider-jack-in-clj (can run a repl without a project)
- supported build tools: clojure-cli, leiningen, babashka, nbb
- jack-in behavior can be customized or over-ridden
- remote and socket connections are possible too
- there are many advanced features possible with additional nREPL
quit: C-c C-q kills REPL

using

activate: with (add-hook ‘clojure-mode-hook #‘cider-mode) or M-x cider-mode (toggle)
evaluate: C-c C-k (buffer), C-c C-e (previous form), C-c C-c (current form)
interrupt: C-c C-b (stuck)
jump to/from: C-c C-z (repl), C-u C-c C-z (match namespace)
ref: C-c C-d C-d|w (docs), M-. (def, but lsp auto-shows)
search: C-c C-d (C-)a (var), C-c C-d (C-)f (apropos doc)

enlive

A selector-based (à la CSS) templating library for Clojure.

group-by problem

stackoverflow tests interesting item to look at Group-by too slow (13 million-lines file) the solution is very instructive and covers generating a test file as well as timing.

hiccup

IO

csv read

(defn csv-read
  "reads a csv file given filename"
  [filename]
  (map s/trim
       (first (with-open
                [in-file (clojure.java.io/reader filename)]
                (doall
                 (clojure.data.csv/read-csv (clojure.java.io/reader in-file)))))))

loop-recur

from nuen 200708

(fn [rs nu-seq]
  (loop [nu nu-seq
         brs 0
         prs 0
         i 0]
    (if (or (> i (- dohlc-size part-size))
            (empty? nu))
      [brs prs]
      (let [nu0 (first nu-seq)
            hi ($ i :h dohlc)
            lo ($ i :l dohlc)
            cl ($ i :c dohlc)
            [is-brs is-prs] (if (= rs "r")
                              [(brs? nu0 hi cl < -) (prs? nu0 hi cl > +)]
                              [(brs? nu0 lo cl > +) (prs? nu0 lo cl < -)])
            ]
        (cond
          is-brs (recur nu (inc brs) prs (inc i))
          is-prs (recur (drop 1 nu) brs (inc prs) (inc i))
          :else (recur nu brs prs (inc i)))))))

project start

Simplest way to start project using cornfeld’s clj-new.

clj -X:new-app :name pradesigner/myapp
clj -X:new-lib :name pradesigner/mylib

after adding to ~/.clojure/deps.edn: ;; add these into your :aliases map:

:new-app {:extra-deps {com.github.seancorfield/clj-new
                       {:mvn/version "1.1.309"}}
          :exec-fn clj-new/create
          :exec-args {:template "app"}}
:new-lib {:extra-deps {com.github.seancorfield/clj-new
                       {:mvn/version "1.1.309"}}
          :exec-fn clj-new/create
          :exec-args {:template "lib"}}}

deps.edn can help you figure out how to do things because of aliases :run-x :uberjar etc contains howto instructions! clj -X:run-x clj -X:uberjar

regex

Clojure Regex Tutorial by Eric Normand

Construct using a literal syntax: #“regex”. Don’t need to double escape #““̣ (match one digit).

Matching a regex to a string with groups

Match an entire string with re-matches.

(re-matches regex string) ;;=> result

;; No match returns nil
(re-matches #"abc" "xyz")            ;;=> nil
(re-matches #"abc" "zzzabcxxx")      ;;=> nil

;; Matching with no groups returns the matched string (truthy)
(re-matches #"abc" "abc")  ;;=> "abc"
(re-matches #"\d+" "3324") ;;=> "3324"

;; Matching with groups returns a vector
(re-matches #"abc(.*)" "abcxyz")       ;;=> ["abcxyz" "xyz"]
(re-matches #"(a+)(b+)(\d+)" "abb234") ;;=> ["abb234" "a" "bb" "234"]

Finding a regex substring within a string with groups

Using re-find returns the first match within the string. It can also be used to see if one string contains another though clojure.string/includes? is likely a better choice.

(re-find regex string)

;; No match returns nil
(re-find #"sss" "Loch Ness") ;;=> nil

;; Match without groups returns the matched string
(re-find #"s+" "mississipi") ;;=> "ss"

;; Match with groups returns a vector
(re-find #"s+(.*)(s+)" "success") ;;=> ["success" "ucces" "s"]

Finding all substrings that match within a string

Return a lazy seq of all of the matches with re-seq.

(re-seq regex string)

(re-seq #"s+" "mississippi") ;;=> ("ss" "ss")
(re-seq #"[a-zA-Z](\d+)"
        "abc x123 b44 234")  ;;=> (["x123" "123"] ["b44" "44"])

Replacing regex matches within a string

Replace all or first substring that matches with some other string with a string or a calculated function using clojure.string/replace.

(clojure.string/replace string regex replacement)

(str/replace "mississippi" #"i.." "obb") ;;=> "mobbobbobbi"
(str/replace "mississippi" #"(i)" "$1$1") ;;=> "miissiissiippii"

(str/replace "mississippi"
             #"(.)i(.)"
             (fn [[_ b a]]
               (str (str/upper-case b)
                    "—"
                    (str/upper-case a)))) ;;=> "M—SS—SS—Ppi" !!!

Splitting a string by a regex

To split a string on some character pattern, like one or more whitespace use clojure.string/split.

(clojure.string/split string regex)

(str/split "This is a string    that I am splitting." #"\s+")
;;=> ["This" "is" "a" "string" "that" "I" "am" "splitting."]

String contains a substring

Various ways to determine if there is a substring in a string.

(re-find #"needle" "Find a needle in a haystack.") ;;=> "needle"
(re-find #"needle" "Empty haystack.")              ;;=> nil

(str/includes? "Find a needle in a haystack." "needle") ;;=> true
(str/includes? "Empty haystack." "needle")              ;;=> false

(re-find #"^This string" "This string starts with ...") ;;=> "This string"
(re-find #"end$" "Find a string at the end")            ;;=> "end"

(str/starts-with? "This string starts with ..." "This string") ;;=> true
(str/ends-with?   "Find a string at the end"    "end")         ;;=> true

Escaping regex

Trying to match a regex symbol containing string can be done with java.util.regex.Pattern and is much preferable to trying to escape escapes (which can’t be done anyway).

"(??^$]" ;; A string I want to match literally
#"\(\?\?\^\$\]" ;; you can't escape the escapes!

(Pattern/quote "(??^$]") ;;=> "\\Q(??^$]\\E"
(-> "(??^$]" Pattern/quote Pattern/compile) ;;=> #"\Q(??^$]\E" !!!

Joining regex

howto combine strings and regex https://stackoverflow.com/questions/21416917/multiple-regular-expressions-in-clojure

(defn union-regex [& patterns]
  \\\"joins regex together\" with |\"
  (re-pattern
   (apply str
          (interpose \""|"
                     (map #(str "(" % ")")
                          patterns)))))
(union-regex #"\w+ " (re-pattern w) " " #"\w+") ;; w set to "how"

=> #”(+ )|(how)|( )|(+)”

or this solution from webscraper project

(defn join-regex [& patterns]
  "joins regex together"
  (re-pattern
   (apply str
          (map #(str %)
               patterns))))
(join-regex #"\w+ " (re-pattern w) " " #"\w+") ;; w set to "how"

=> #"+ how +"

Extract whole sentences with matching word

https://stackoverflow.com/questions/26081820/regular-expression-to-extract-whole-sentences-with-matching-word [^.]* flung [^.]*\ [^.?!]*(?<=[.?!])flung(?=[.?!])[^.?!]*[.?!]

scrapers

string

clojure.string functions

blank?
capitalize
ends-with?
escape
includes?
index-of
join
last-index-of
lower-case
re-quote-replacement
replace
replace-first
reverse
split
split-lines
starts-with?
trim
trim-newline
triml
trimr
upper-case

official clojure site

Use clojure.org as a reference for working with Clojure, but also put in specifics about own setup and tools.

Overview

Why Clojure?
Lisp is a good thing
Functional programming is a good thing
Languages and Platforms
Object Orientation is overrated
Polymorphism is a good thing
Concurrency and the multi-core future

Guides

LEARNING
- Getting Started
  - emacs editor
  - eglot Eglot is the Emacs client for the Language Server Protocol (LSP), which provides advanced programming language support and features for code editing in Emacs. It can do Language Server Integration, Auto-Completion, Go to Definition (M-.), Find References (M-?), Error Highlighting and Code Diagnostics (with flymake), Signature Help, Refactoring Support.
  - clojure-lsp don’t need to use it directly as a mode because eglot looks after that, but it does need to be installed. For instance, Find references didn’t work when it was not installed. Perplexity got this wrong.
  - cider see details in cider section and look in the manual.
- Install Clojure
- Editors
- Structural Editing
REPL Programming
- Introduction
- Launching a basic REPL
- Basic Usage
- Data Visualization at the REPL
- Navigating namespaces
- Enhancing your REPL workflow
- Guidelines for REPL-Aided Development
- Community resources
- Troubleshooting
Learn Clojure
- Introduction
- Syntax
- Functions
- Sequential Collections
- Hashed Collections
- Flow Control
- Namespaces
FAQ
LANGUAGE
- spec
- Reading Clojure Characters
- Destructuring
- Threading Macros
- Equality
- Comparators
- Reader Conditionals
- Higher Order Functions
USAGE
- Dev Startup Time
TOOLS
- Deps and CLI ~/.clojure/deps.edn (global deps.edn no :deps)
  clj -X:deps find-versions :lib clojure.java-time/clojure.java-time
- tools.build
LIBRARIES
- core.async Walkthrough
- Go Block Best Practices
- test.check

Reference

Reader forms
Macro characters
Tagged Literals
Reader Conditionals

Community

Success Stories
Community Stories
Companies
Tools
Libraries
Contributing
Community Resources
Forum
Events
Training
Etiquette
HELP
Books
User Groups
Starting a User Group
SITE UPDATES
Site Contributions
Editing tips
License
Privacy Policy
ETC
Swag

API

Clojure API
ClojureScript API
Clojure Contrib Libraries
Other resources: ClojureDocs and cljdoc

DEV

LANGUAGE DEV
Development
Contributor Agreement
Workflow
Creating Tickets
Developing Patches
Screening Tickets
Contributors
CONTRIB LIBS
Contrib libs
Contrib lib history
RESOURCES
Source
Support Portal
Issue Tracker
Downloads
API Docs
License

Releases

CLOJURE
Current Release
Older Releases
Dev Changelog
Download key
CLOJURE TOOLS
Tools Releases

News

Ongoing and updated regularly.

Clojure website Clojure Documentation clojure cheatsheet CIDER ClojureDocs ask.clojure forum Pedestal kixi stats

websites

CIDER
Clojure(Script) Interactive Development Environment that Rocks extends Emacs with support for interactive programming in Clojure. The features are centered around cider-mode, an Emacs minor-mode that complements clojure-mode and clojure-ts-mode. While clojure-mode supports editing Clojure source files, cider-mode adds support for interacting with a running Clojure process for compilation, debugging, definition and documentation lookup, running tests and so on.
ClojureDocs
ClojureDocs is a community-powered documentation and examples repository for the Clojure programming language.
Clojure Q & A
At Clojure Q&A, you can ask questions and receive answers from members of the Clojure community.
Pedestal
A set of libraries that to build web services and applications. It runs in the back-end and can handle anything from tiny static web sites, to traditional page oriented applications, to dynamic single page applications utilizing server-sent events and WebSockets.
kixi.stats
A Clojure/ClojureScript library of statistical sampling and transducing functions.
Clojure Toolbox
A categorised directory of libraries and tools for Clojure.
Clojars
This is an easy to use community repository for open source Clojure libraries.

videos

zz

ideas

do programming with awareness of each function’s io

snippets

alternate possibilities with or

(defn mk-tohlc
  "creates the dataset with date, open, high, low, close from filename"
  [fpath]
  (let [time (or "t" "column-0")
        open (or "o" "column-1")
        high (or "h" "column-2")
        low (or "l" "column-3")
        close (or "c" "column-4")]
    (rename-columns
          (->dataset fpath {:header-row? false
                            :parser-fn {time :string
                                        open :float32
                                        high :float32
                                        low :float32
                                        close :float32}})
          {time :time open :open high :high low :low close :close})))

files

Listing files in a directory in Clojure

(seq (.list (clojure.java.io/file ".")))
(map #(.getPath %)
     (file-seq (clojure.java.io/file ".")))

file

fs - File system utilities for clojure https://github.com/Raynes/fs

;using regex

(def files
  (filter #(re-find #"1D|4H" %)
          (map #(.getPath %)
               (file-seq (clojure.java.io/file "/zata/truefx")))))

;using string

(def files 
  (filter #(clojure.string/includes? % "1D")
          (map #(.getPath %)
               (file-seq (clojure.java.io/file "/zata/truefx")))))

;going through a vector

(def files
  (map (fn [x]
         (filter #(clojure.string/includes? % x)
                 (map #(.getPath %)
                      (file-seq (clojure.java.io/file "/zata/truefx")))))
       ["1D" "4H" "1D"]))

files from filepaths

(def filepaths
  "produces a list of truefx path strings"
  (flatten (map (fn [x]
                  (sort (filter #(str/includes? % x)
                                (map #(.getPath %)
                                     (file-seq (clojure.java.io/file "/zata/truefx"))))))
                ["1D" "4H" "1H"])))

(def filenames
  "extracts file names from file paths"
  (map #(last %)
       (map #(str/split % #"/")
               filepaths)))

partitions and maxmin

since partition gives list, we need to apply max to each list we map the apply max over the partitions

(map #(apply max %) (partition 3 r))

removing nil

(filter #(not(nil? %)) [1 nil 2 3 nil 5])
(1 2 3 5)
(remove nil? [1 nil 2 3 nil 5])
(1 2 3 5)

finding nu

(defn nusr
  "identifies sr-levels for instrument using nu"
  [fp]
  (let [[o h l c fn pf] (inst-info fp)
        part-size 5
        step-size 1
        midpt (quot part-size 2)
        hi (partition part-size step-size h)
        lo (partition part-size step-size l)
        n (remove nil? (map #(nu % max midpt) hi))
        u (remove nil? (map #(nu % min midpt) lo))]
    [:ns n :us u]))

(defn nu
  "finds a nu given seq, maxmin, midpt of seq"
  [sq maxmin midpt]
  (let [midval (nth sq midpt)]
    (when (= (apply maxmin sq) midval)
      midval)))

(nusr (first truefx-filepaths))

combining vectors

problem came up trying to work with bounsr and penesr where we needed to take 2 vectors and generate a vector of vectors matching items in sequence:

[4 4 6] [7 8 9] => [[4 7] [4 8] [6 9]]

(zipmap [4 4 6] [7 8 9]) => {4 8, 6 9} due to repetition of the 4 because it is a map

this thread Is there an equivalent for the Zip function in Clojure had 2 solutions:

(map vector [4 4 6] [7 8 9]) => ([4 7] [4 8] [6 9]) (partition 2 (interleave [1 2 3] [4 5 6])) => ((1 4) (2 5) (3 6))

(interleave [1 2 3] [4 5 6] [7 8 9]) (apply interleave [[1 2 3] [4 5 6] [7 8 9]])

or as a function

(defn zip [& colls] (partition (count colls) (apply interleave colls)))

diff btn map and apply

(apply F [1 2 3 4 5]) => (F 1 2 3 4 5) (map F [1 2 3 4 5]) => [(F 1) (F 2) (F 3) (F 4) (F 5)] https://stackoverflow.com/questions/2311528/clojure-apply-vs-map

Use a vector as a LIFO stack to check for balanced brackets

(require '[clojure.set :refer [map-invert]])

(defn balance [form]
  (let [brackets {\[ \] \( \) \{ \}}
        scan (fn [q x]
               (cond
                 (brackets x) (conj q x)
                 ((map-invert brackets) x)
                 (if (= (brackets (peek q)) x)
                   (pop q)
                   (throw
                     (ex-info
                       (str "Unmatched delimiter " x) {})))
                 :else q))]
    (reduce scan [] form)))

(balance "(let [a (inc 1]) (+ a 2))")
ExceptionInfo Unmatched delimiter ]

(balance "(let [a (inc 1)] (+ a 2))")
[]

dataset vs matrix

(type (incanter/to-matrix ds)) mikera.arrayz.impl.JoinedArray or vectorz/matrix (type (m/array ds)) clojure.lang.PersistentVector


cmd	dataset	matrix	array	lazyseq
head,tail	y	y	y	n
take,drop	y	y	y	y

getting stuff out of incanter dataset dohlc

clojure.core.matrix.impl.dataset.DataSet

using $ producing lazyseq

his ($ :h dohlc) clojure.lang.LazySeq or (sel dohlc :cols :h) los ($ :l dohlc) clojure.lang.LazySeq or (sel dohlc :cols :l)

using to-map producing persistent vectors

ds (d/to-map dohlc) clojure.lang.PersistentArrayMap his (:h ds) clojure.lang.PersistentVector los (:l ds) clojure.lang.PersistentVector

using m/array producing vectorz

ds (d/to-map dohlc) clojure.lang.PersistentArrayMap his (m/array (:h ds)) mikera.vectorz.Vector los (m/array (:l ds)) mikera.vectorz.Vector can’t (m/array ds) because clojure.lang.PersistentArrayMap cannot be cast to java.lang.Number though (type (m/array d)) clojure.lang.PersistentVector

sel vs $

it seems as though we can use $ for pretty well everything we use sel for (sel d :cols :h) <=> ($ :h d) (sel d :rows (range 2 11) :cols :d) <=> ($ (range 1 11) :d d) (sel d :rows 1 :cols :h) <=> (sel d :rows 1 :cols 2) <=> (sel d 1 2) <=> ($ 1 2 d) <=> ($ 1 :h d)

codeOLD

the ratios function doesn’t produce same results as the individual codings we haven’t figured out why or which is really correct (defn anulysisV2 " analyzes brs and prs price action on nu

+brs? (fn [nu hilo cl >< -+] “n: (brs? n hi cl < -) u: (bsr? u lo cl > ~~)” (and (btn? hilo (- nu buffer) (~~ nu buffer)) (>< cl (-+ nu buffer)))) prs? (fn [nu cl hilo >< -+] “n: (prs? n hi cl > ~~) u: (prs? u lo cl < -” (and (>< hilo (-~~ nu buffer)) (>< cl nu)))

ratios (fn [rs nu-seq] (loop [nu nu-seq brs 0 prs 0 i 0] (let [nu0 (first nu-seq) hi ($ i :h dohlc) lo ($ i :l dohlc) cl ($ i :c dohlc) [is-brs is-prs] (if (= rs “r”) [(brs? nu0 hi cl < -) (prs? nu0 hi cl > +)] [(brs? nu0 lo cl > +) (prs? nu0 lo cl < -)]) ] (if (> i (- dohlc-size part-size)) [brs prs] (cond is-brs (recur nu (inc brs) prs (inc i)) is-prs (recur (drop 1 nu) brs (inc prs) (inc i)) :else (recur nu brs prs (inc i))) ))))

n-ratio (loop [nn nseq brs 0 prs 0 i 0] (let [firstnn (first nn) hi ($ i :h dohlc) cl ($ i :c dohlc)] (if (> i (- dohlc-size part-size)) [brs prs] (cond (brs? firstnn hi cl < -) (recur nn (inc brs) prs (inc i)) (prs? firstnn hi cl > +) (recur (drop 1 nn) brs (inc prs) (inc i)) :else (recur nn brs prs (inc i))) ))) u-ratio (loop [uu useq brs 0 prs 0 i 0] (let [firstuu (first uu) lo ($ i :l dohlc) cl ($ i :c dohlc)] (if (> i (- dohlc-size part-size)) [brs prs] (cond (brs? firstuu lo cl > +) (recur uu (inc brs) prs (inc i)) (prs? firstuu lo cl < -) (recur (drop 1 uu) brs (inc prs) (inc i)) :else (recur uu brs prs (inc i))) ))) n-ratio (ratios “r” nseq) u-ratio (ratios “s” useq) ] [n-ratio u-ratio] ))

while the ideas here are ok we require 20s by not using partitions and 25ms when we do!! however, we need a way to figure out the index

(defn nu-finderV2 " finds nu give a dohlc dataset returns [[i nn] [i uu]] " [dohlc] (let [size (nrow dohlc) nu-breadth 7 ;nu width buf-bars (quot nu-breadth 2) ;number of bars on either side of nu ibeg buf-bars ;begining bar to look for nu iend (- (nrow dohlc) 1 buf-bars) ;ending bar to look for nu nu? (fn [i hilo maxmin] (= ($ i hilo dohlc) (apply maxmin ($ (range (- i buf-bars) (+ i buf-bars 1)) hilo dohlc)))) nus (fn [hilo maxmin] (loop [i ibeg iv []] (if (>= i iend) iv (if (nu? i hilo maxmin) (recur (+ i buf-bars 1) (conj iv [i ($ i hilo dohlc)])) (recur (inc i) iv)))))] [(nus :h max) (nus :l min)]))

(defn anulysisV1 " analyzes brs and prs price action on nu

dohlc dataset columns :d date :o open :h high :l low :c close identify nu using partition map bsr? psr? to hilo and determine how many of each " [filepath] (let [[dohlc filename pipfactor] (instrument-dohlc filepath) dohlc-size (nrow dohlc) ;size of dataset less 2 because of header buffer (/ 12.0 pipfactor) ;buffer line calculation in pips for pair part-size 7 ;size of partition chosen nn (nu-finder ($ :h dohlc) part-size max) uu (nu-finder ($ :l dohlc) part-size min) in-zone? (fn [hilo nu] (btn? hilo (- nu buffer) (+ nu buffer)))] (loop [i 4 brs 0] (if (> i (- dohlc-size 9)) brs (recur (inc i) (if (in-zone? ($ i :h dohlc) (first nn)) (inc brs) brs))) ) [(count nn) (count uu)]))

(defn mk-dohlc “creates the dataset with date, open, high, low, close from filename” [filename] (d/rename-columns (read-dataset filename) {:col0 :d :col1 :o :col2 :h :col3 :l :col4 :c}))

(defn nu-finderV1 " finds all nu in a sequence of hilo given a sequence, the partition size, max|min operator for his|los TODO have option to remove invalid nu " [aseq part-size maxmin] (let [partitions (partition part-size 1 aseq) midpt (quot part-size 2) nu (for [p partitions :when (= (apply maxmin p) (nth p midpt))] (nth p midpt))] nu))

(def truefx-filepaths “produces a list of truefx path strings” (vectorize (flatten (map (fn [x] (sort (filter #(str/includes? % x) (map #(.getPath %) (file-seq (clojure.java.io/file “/zata/truefx”)))))) [“1D” “4H” “1H”]))))

(defn adjacent-confluence “determines oc confluence behavior of adjacent bars” [fp] (let [dohlc (mk-dohlc fp) fn (filename-from-filepath fp) pf (pip-factor fn) m (to-matrix (ds/remove-columns dohlc [:d])) o (sel m :cols 0) h (sel m :cols 1) l (sel m :cols 2) c (sel m :cols 3) co (map pips c o (repeat pf)) pn (map #(pos? (* %1 %2)) co (drop 1 co)) p% (Math/round (/ (count (filter true? pn)) (float (count pn)) 0.01))] (printf “%s %d %n” fn p%) p%))

to-solve

match a key and print component of a map

here is a vector of maps (def asym-parts [{:name “head” :size 3} {:name “mouth” :size 1} {:name “nose” :size 1} {:name “neck” :size 2} {:name “left-eye” :size 1} {:name “left-ear” :size 1}to {:name “left-shoulder” :size 3}])

match the :name to be “nose” and print that item TODO

an interesting solution to something similar https://stackoverflow.com/questions/18176372/clojure-get-map-key-by-value

(->> {:a “bar” :b “foo” :c “bar” :d “baz”} ; initial map (group-by val) ; sorted into a new map based on value of each key (#(get % “bar”)) ; extract the entries that had value “bar” (map key)) ; get the keys that had value bar or (map key (#(get % “bar”) (group-by val {:a “bar” :b “foo” :c “bar” :d “baz”})))

old diary items

<2020-07-24 Fri> figured out how to install own work to .m2 started rewrite of algolyze using these ideas <2020-07-23 Thu> went through clojure style guide got better understanding of ns installed and worked with hylang - though we should check out cljpython too at some point

discussion on tech.ml was very promising with both gigasquid and joinr praising it. so that’s what we’ll go with for now - convert existing items to it and look for a visualization mechanism. <2020-07-22 Wed> got nuen.clj v1 done and testing.

<2020-07-18 Sat> go thru clojure reader is unsafe post about times post about project mechanism

<2020-07-17 Fri> nuen mostly done thx to help on clojurians

techml read into techml dataset a csv file nearly 3 million lines drop one column form new column the result of subtracting one column from another printout head of adjusted dataset

timings: lein run 62.93s user 0.86s system 123% cpu 51.800 total emacs repl “Elapsed time: 40112.441757 msecs” java -jar target/uberjar/techml-0.1.0-SNAPSHOT-standalone.jar 29.95s user 0.77s system 162% cpu 18.916 total

<2020-07-16 Thu> start nuen

rewrite anulysis for tech.ml rewrite anulysis for arrays

<2020-07-15 Wed> finish mikera and incanter build a matrix, dataset, array toolkit (mikera incanter and anything else) plan for clj cheatsheet explorations -> make own cheatsheet from it and others plan for java function access

will conversion to separate arrays be more efficient for subsequent processing? using arrays instead of incanter datasets may be practical?

<2020-07-14 Tue> finishing mikera and incanter we should go through cheatsheet items and understand how the main functions in the categories work

<2020-07-13 Mon> last couple of days were spent with julia programming. though the docs are excellent, emacs doesn’t work too well with it right now. emacs does work well with R. so may be …

attempt to get python libraries on clojure using - but carin’s example doesn’t work completely. anyway, we don’t really need to do this now because what we were interested in seeing was how well anulysis worked and we can actually do this better in other ways - such as plotting hlc dots with buffers from nu

checked out graalvm and that does look very interesting! again, we don’t need it now, but it is definitely something to keep in mind for the future considering one demo showed 20 performance improvement. it may be useful even without its polyglot capabilities.

for now, here’s what we focus on:

learn mikera and incanter
learn what we can do with direct access to java
optimize anulysis
get nuen completed

<2020-07-10 Fri> got R and candlesticks working - it’s pretty good! main difficulty was that manjaro doesn’t autoload make and gcc-fortran

plotly.clj doesn’t seem to have candlesticks plan is to go through incanter plotting and compare to plotly.clj plotting

<2020-07-09 Thu> anulysis finish hopefully

plan for manual trading

examine situation around 12am
put in suitable entry limit or stop trades with sltp
try to exit trade around 12pm next day
close orders and start at 1 again

<2020-07-08 Wed> cider site debugging cider debugging tools debugging in clojure several items not quite understood but we now have a useable debugging mechanism

more learnt about loop-recur in that the main if statement can have various forms (eg let) in true or false. this knowledge enabled the best version to-date: (fn [rs nu-seq] (loop [nu nu-seq brs 0 prs 0 i 0] (if (or (> i (- dohlc-size part-size)) (empty? nu)) [brs prs] (let [nu0 (first nu-seq) hi ($ i :h dohlc) lo ($ i :l dohlc) cl ($ i :c dohlc) [is-brs is-prs] (if (= rs “r”) [(brs? nu0 hi cl < -) (prs? nu0 hi cl > +)] [(brs? nu0 lo cl > +) (prs? nu0 lo cl < -)]) ] (cond is-brs (recur nu (inc brs) prs (inc i)) is-prs (recur (drop 1 nu) brs (inc prs) (inc i)) :else (recur nu brs prs (inc i)))))))

<2020-07-07 Tue> getting nu the partition mechanism provides far faster results than iterating over groups of 7 bars - even if you skip bars regularly because after a nu the latter three can never be a nu. however, the partitions don’t provide anyway of keeping track of the index. a problem can develop in that as we go through nu we could be counting brs even before the occurence of the nu!

so here are 2 ideas: figure out the index of the nu through a loop-recur setup. something like loop ds to find index conj index+accumulatedindices to nu drop index+1 ds

similar idea, but we don’t need to actually put the index anywhere. find index of nu drop to index+1 start to check for brs once prs drop current nu do over again this way we only start checking for brs after we find where the nu is

<2020-07-06 Mon> figured out how to use loop-recur better - it is a likeable combo and seems preferable to me than recursion now. got a good version of nu-finder done which includes index amongst other things and can be used outside anulysis. <2020-07-05 Sun> setup nu-orders.clj for populating nu with orders <2020-07-04 Sat> finish anulysis <2020-07-03 Fri> go through google hits example

<2020-07-02 Thu> determine whether to use core.matrix or incanter for most things incanter is being incorporated within core.matrix and it is not really developed much. so we’ll make the switch and start looking at some of the other data science libraries.

discovered core.matrix creator mike anderson or mikera who has good youtube videos on the topic

seems as though core.matrix (most operations) and incanter (rw graphics etc) can work well together.

<2020-07-01 Wed> issues with datasets and matrices using map and apply figured out a lot about for loop <2020-06-30 Tue> dilemmas with figuring out how to work hilo with nus <2020-06-29 Mon> figuring out a more iterative anulysis <2020-06-28 Sun> work on sicp - understanding cond and if <2020-06-27 Sat> started sicp and INDUCTIVESYNTHESISOFFUNCTIONALPROGRAMS

<2020-06-26 Fri> figure out mechanism to compare hilo to stacked nu problem is that i keep wanting to put things into boxes rather than thinking functionally.

<2020-06-25 Thu> work on anulysis using stacks and proper influence.

(def d (ds/select-rows ds (range 9))) (ds/add-column d :sums (mt/add (sel d :cols :o) (sel d :cols :h))) (ds/add-column d :sums (mt/gt (sel d :cols :c) 83)) (ds/add-column d :sums (mt/mul (sel d :cols :c) (mt/gt (sel d :cols :c) 83))) ;crude

<2020-06-24 Wed> creating bounsr and penesr functions within let got involved with creating pairs

though there are some nice ideas in V1 of anulysis the overall attempt to have nu checked over the entire matrix is meaningless and produced absurdly high numbers - eg almost 8000 bounsr for nrow = 2248. so we need to restrict the nu influence to forward items only and have them flip after first penesr and disappear after a second penesr.

(defn anulysisV1 “analyzes pa near nu” [fp] (let [[ops his los cls fnm pf] (instrument-info fp) buffer (/ 12 pf) bounsr? (fn [nus hilos cls >< -+] (count (remove nil? (for [nu nus [hilo cl] (zip hilos cls)] (let [nu-buffer (-+ nu buffer)] (if (and (btn? hilo [nu nu-buffer]) (>< nu-buffer cl)) true)))))) bpsr? (fn [nus hilos ops cls >< -+] (for [nu nus [hilo op cl] (zip hilos ops cls)] (let [b-buffer (-+ nu buffer) p-buffer (-+ nu (- buffer))] (cond (and (btn? hilo [nu b-buffer]) (>< b-buffer cl)) :bounsr (and (btn? nu [op cl]) ) :penesr)))) part-size 5 midpt (quot part-size 2) hi-partition (partition part-size 1 his) lo-partition (partition part-size 1 los) nn (remove nil? (map #(nu-finder % max midpt) hi-partition)) uu (remove nil? (map #(nu-finder % min midpt) lo-partition)) boun-s (bounsr? uu los cls < +) boun-r (bounsr? nn his cls > -)

isbounsr #(= :bounsr %) ispenesr #(= :penesr %) bpu (bpsr? uu los ops cls < +) bpn (bpsr? nn his ops cls > -) ] [fnm boun-s boun-r (count (filter isbounsr bpu)) (count (filter isbounsr bpn)) (count (filter ispenesr bpu)) (count (filter ispenesr bpn))]))

(map anulysis (take 1 truefx-filepaths)) => ([“audjpy1D.csv” 3829 3607]) => ([“audjpy1D.csv” 4167 4035])

<2020-06-23 Tue> coined term lalwal lose a little win a lot rewrote outliers in truefx files - all were with the lows. some involving CHF may have been legitimate. anulysis almost done after figuring out that :while is not to be used with for - not sure what it is

<2020-06-22 Mon> started anulysis

<2020-06-21 Sun> get barchar working first idea of adjacent-confluence may work in reverse for shorter tf

started snippets in this file

made neat ns discovry - whatever is in core.clj including def/defn becomes available to every other file provided the namespace isn’t changed! this really means that we can load up core.clj with whatever is required rather than import everything for every related program we create!

<2020-06-20 Sat> working with dataset functions add-column, reorder-column, remove-column wrote sel-col function which is useful to work with single column in dataset

discovered that sel on a dataset is very slow compared to sel on a matrix it seems to be far, far faster to convert a dataset to a matrix to do things and then convert it back if you want to utilize the variety of types a dataset offers. (see xpts in datmat.clj)

<2020-06-19 Fri> investigating various things all week without entry into diary many of clojure.core.matrix is replacing the incanter functions: see updates to incanter 2.0 https://github.com/incanter/incanter/wiki/Incanter-2.0-change-log so we investigate these in datmat.clj

<2020-06-11 Thu> get-orders-for done rearrange the ztut which was getting awkward as far as ns goes as well as other things. set it all up on gitlab

changing to ns works with C-c M-n after we C-c C-k the buffer. for some reason, (in-ns namespace) doesn’t always work. this page is useful as may be other things on that site.

<2020-06-10 Wed> incanter things work nicely so far, but we need to become far more knowledgeable working with datasets, functions (what they do) and plotting.

also, the how i start thing is a bit confusing as carin jumps to things that we’re not familiar with so we’ll get back to that later.

<2020-06-09 Tue> incanter:

work through data sorcery
- cheatsheet use regularly to recall syntax and concepts
- make use of api documentation throughout
- namespace switching solved: repl takes main ns, but can switch with (in-ns ‘whatever) so long as it is seen
started How I Start which should be useful for namespace understanding and testing

<2020-06-08 Mon> incanter:

work through ch6 Working with Incanter Datasets from Clojure Data Analysis Cookbook which was helpful though the datasets used were huge and therefore difficult to examine in some situations

<2020-06-07 Sun> installed inlein started thinking bayes

<2020-06-06 Sat> transferred from github to gitlab which seems to be smoother and cleaner. cleaned up tutorials into ztut folder cleaned up clj bookmarks went through excellent birdseye view of clj A Brief Beginner’s Guide To Clojure

<2020-06-05 Fri> more cider it seems that form order is important because loading the buffer when dependencies don’t come first makes eval impossible. cider has a lot of functions, but we need to figure out which ones we’ll actually use successfully connected to an existing repl at localhost after getting port using: connection info C-c M-d

paredit:


M-( M-"	M-s M-?	C-right C-left	M-S M-J	C-M-f C-M-b

cider:


C-c C-z	C-c M-j	C-c C-c	C-c C-k	C-up C-down

<2020-06-04 Thu> got all of oanda ide to date as far as we had python going order fn greatly simplified - we never needed to use if-let

investigated cider further watched alchy bozbat

<2020-06-03 Wed> stuck on json not being parsed but that was because clojure follows the curl more closely using :body instead of :json whereas python didn’t

<2020-06-02 Tue> get api functions and sring theory to where we were with python all api done as far as python progress except for the json issue

hobbit problem using reduce is neatest

excellent video with edmund jackson data science and clojure shows the landscape in 2012 and what options there were then probably worth following up on this.

<2020-06-01 Mon> understood the hobbit setup mostly in brave it is messy using loop, but much messier other ways

<2020-05-31 Sun> set up tut structures in clojure directory set up problems to solve file in to-solve.clj file

Clojure Libs and Namespaces: require, use, import and ns (require ‘clojure.string ‘clojure.test) (require ‘[clojure.string :as string]) to avoid (require [‘clojure.string :as ‘string]) (require ‘clojure.string ‘[clojure.test :as test]) (require ‘(clojure string test)) (require ‘(clojure [string :as string] test))

consider the incanter lib issue (use ‘(incanter core stats charts io)) works right away and we can call view, histogram etc (require ‘(incanter core stats charts io)) doesn’t seem to work at all, but that’s due to ns issues: (incanter.core/view (incanter.charts/histogram (incanter.stats/sample-normal 1000))) is just fine

also see Clojure Namespaces and Vars for better ways as well as this answer from stackoverflow. we can solve this with (ns tut-incanter.core (:require (incanter [core :refer :all] [stats :refer :all] [charts :refer :all]))) though (use ‘(incanter core stats charts)) seems much simpler

both api requests with clj-http and json readability accomplished next step is to get functions to work

<2020-05-30 Sat> figuring out git mostly getting through brave ch3 except last exercise

<2020-05-29 Fri> started clojure for brave updated several youtube resources

<2020-05-28 Thu> finished cds introduction

<2020-05-27 Wed> halfway through cds introduction why clojure video by vijay is useful

<2020-05-26 Tue> continued with cds set up resources and videos watched russ olsen functional programming in 40 min

<2020-05-25 Mon> set up emacs for clojure watched rick hickey video on creating it got cider and lein working started cds tutorial

Emacs

Thu, 16 Jun 2022 12:40:30 -0700

We belong to the church of emacs which we feel is the ultimate editor!

keybindings

assistance

The emacs help manual and help summary TODO lots to check out here
Gateway to finding anything on emacs is C-h ?
Regexps can be used as lists of words which are ANDed.
Some prefix followed by C-h give list of bindings for prefix.

;; quick help
C-h C-q

;; apropos
M-x apropos Display all symbols matching a pattern
M-X Display all commands possible for buffer
C-h a Search only commands (apropos-command)
C-h d Search docs (apropos-documentation)
M-x apropos-library Useful for new mode or package
M-x apropos-value Useful if you know value not variable

;; describes
C-h m Mode major/minor, keybindings (describe-mode)
C-h x Commands (describe-command)
C-h f Functions (describe-function)
C-h v Variable (describe-variable)
C-h k Key detail (describe-key)
C-h c Key brief (describe-key-briefly)

All describe buffers have shortcut keys i (info), s (source), c (customize).

;; info
C-h i Open info manual
C-h i d m <item> RET i <topic> RET Indices search
C-h i d m <item> RET s <topic> RET Regexp search
C-h r m <topic> Direct way to emacs info
M-x info-apropos searches for item in all info
M-x info-display-manual get manual by name (or tab for list)
M-x info-finder uses list of keywords to find things

;; various
C-h p Packages list
C-h e Messages buffer
C-h o Symbol docs at point or type it
C-h s Current syntax table !!!

M-x manual-entry RET manpage Get a manpage
<prefix> C-h all prefix possibilities

bookmarks

C-x r m set bookmark
C-x r b jump to bookmark
C-x r l list bookmarks
M-x bookmark-delete

buffers

C-x b helm-buffers
C-x C-b helm-multi-files
C-x M-b ibuffer
C-x k kill buffer

case

M-c Word capitalize
M-u Word ucase
M-d Word dcase
C-x C-u Region ucase
C-x C-l Region lcase
M-x upcase-initials-region Which is strange for most situations.

comment

M-; Comment single/double/region/uncomment
C-x C-; Double comment (lisp)/uncomment
M-x comment-box

Various variables can be customized as well to suit requirements.

count

M-= Words (buffer or region)
M-x count-lines-region
M-x count-matches

dired

Only most useful bindings here. (See dired section for more)

;; commands on files
! Any command on a file
! cmd * > newfile will have output of cmd on * (cursor file)

;; compare files
=  goto file, =, enter other file (figure out how diff works !!!)

;; enter a dir
C-x (4|5) d ~/ new window|frame
C-x d ~/*|.*|D* all|hidden|starting with D files

;; markings
m mark
d mark for deletion
t toggle marks
u,U unmark
M-del unmark all
%d regexp for deletion
~|# bkp|auto-save for deletion
%m regexp flag
%g regexp contained in files
*s|/|.|@ mark all|dirs|ext|sym

;; move
j move to file
(M-)C-s (regexp)search
M-{|} up|down to marked
^ move to parent dir
<|> move to prev|next dir

;; specific commands on marked items
C|D|P|R copy|delete|print|rename
G|M|O dired-do-chgrp|chmod|chown
A|Q regexp search|query replace
!|& shell command on files sync|async
%u|l upper|lower
S|Y|H links sym|relsym|hard

;; subdirs
i insert subdir
C-u k remove subdir
$ fold subdir
M-$ fold all dirs
C-M p|n go to prev|next subdir
C-M-d go to first child subdir


;; view directory
C-u s change ls flags
i put cursor dirs into same buffer

;; visit
e|f same window find
o other window find (C-o don't move there)
v read only
^ parent dir
a descend discarding parent  
W guess default application

;; zzmisc
h help
? prompt for choice delete,copy,find ...
B|L byte-compile|load elisp files

k dired-do-kill-lines don't delete !!!

greps

;; rg
C-c s gives menu to do various greps

;; helm ag
M-g s helm-do-grep-ag (alt to C-s)

;; grep
M-x grep Prompts for item and files
M-x grep-find Uses find and -exec grep
M-x lgrep Prompt through each step
M-x rgrep Recursive search

helm

C-x c (prefix) C-g (exit usually)

;; helmed key bindings
M-x helm-M-x
M-y helm-show-kill-ring
M-s o helm-occur
C-x C-f helm-find-files
C-x C-b helm-multi-files (use to switch buffers)
C-spc mark buffers
M-D delete marked buffers
C-c g helm-google-suggest
M-g f helm-do-grep-ag (alt to C-s)

;; helm prefixed
/ helm-find files recursively from pwd
@ helm-list-elisp-packages
f helm-multi-files buffers,recent,bookmarks,pwd (C-x C-b)
l helm-locate locate through helm
r helm-regexp build regexp for find in file

;; help indices
h g helm-info-gnus
h h helm-documentation
h i helm-info-at-point
h r helm-info-emacs

filling

M-p Fill paragraph
C-u M-q Fill and justify
C-x f Set fill width
C-x . Set fill prefix (for each fill line)
M-x auto-fill-mode (likely best for text production)

insert

C-x i Insert file
C-x x i Insert buffer (since all buffers aren't files)

kill

M-d Word forward from point
C-<back> Word backward from point
C-k To end of line
C-S-k To beg of line
M-k Rest of paragraph

M-y TAB will show kill ring in buffer for yank.

macro

F3 record beg
C-u F3 record beg and append to last
F4 record end | play last
C-u F4 play 2nd
M-<N> F4 play last N times

C-x C-k n names last
C-x C-k b binds last to key
M-x insert-kbd-macro pastes into buffer
C-x C-k e edits last

mark

M-@ Word (M-- backward)
M-h Paragraph (M-- backward)
C-x h Entire buffer
C-x C-p Next page

in ibuffer
% n - Mark buffers by their name, using a regexp.
% m - Mark buffers by their major mode, using a regexp.
% f - Mark buffers by their filename, using a regexp.
% g - Mark buffers by their content, using a regexp.
% L - Mark all locked buffers.

Often unnecessary, except copying region or kill/copy weird shaped regions.

move

;; smaller movements
M-m Line beg (regardless of indent)
C-a Line beg
C-e Line end
M-a Sentence beg
M-e Sentence End
M-down Pargraph next beg
M-up Paragraph next end

;; larger movements
M-i Use imenu
M-r Cycle position point bot-mid-top
C-l Cycle center point to bot-mid-top
M-g M-g go to line number

;; sexp movements!!!
C-M-d Down into list
C-M-u Up out of list
C-M-n Forward to next list
C-M-p Backward to prev list
C-M-h Next defun
C-M-@ Next sexp
C-M-f Forward by sexp
C-M-b Backward by sexp
C-M-a Descend into sexp

negate

C--
M--
C-M-

M-- M-d kills word before point

search/replace

C-s|r Use isearch forward|reverse
C-M-s|r Use isearch regexp
M-% Query search options
C-M-% Query regexp options
M-s Replace string
M-r Replace regexp
!!! c|r|w|_|'|SPC Toggle various modes!!!
!!! C-w Add word at point to search (??!!!)

Note case folding effect. Regexps are fine for simple search/replace, but for heavy-duty stuff emacs/elisp is likely not the best option. So, we should use perl or clojure from the outside.

tab bar mode

C-x t 2 create a new tab
C-x t 0 close current tab
C-x t RET Select tab by name 
C-x t o Next Tab (C-<tab>)
C-S-<tab> Previous Tab 
C-x t r Rename Tab 
C-x t m Move tab one position to right 
C-x t p ... Run project command in other tab 
C-x t t Execute command in other tab 
C-x t 1 Close all other tabs 
C-x t C-f Find file in other tab 
C-x t b Switch to buffer in other tab 
C-x t d Open Dired in other tab
M-x tab-list Shows an interactive tab list 
M-x tab-undo Undoes a closed tab for each invocation 
M-x tab-recent Switch to the last visited tab
M-x tab-bar-history-mode
M-x global-tab-line-mode

text manipulation

;; duplicates in region
M-x delete-duplicate-lines With 3 C-u options

;; keep, flush, copy in region by regexp
M-x flush-lines
M-x keep-lines
M-x copy-matching-lines
M-x kill-matching-lines

;; kill
C-k kill to end of line
C-; kill to beg of line

;; occur
M-x occur Choose e option to edit

;; join and split lines
C-o Inserts blank line after point with possible split
C-M-o Splits line at point keeping indent
C-x C-o Deletes all blank lines after point
M-^ Joins with line above

;; whitespace
M-SPC Cycle spacing instead of just-one-space
M-\ Deletes whitespace around point
M-x delete-trailing-whitespace
M-x whitespace-report(-region)
M-x whitespace-cleanup(-region)

transpose

C-t Characters
M-t Words
C-x C-t Lines
C-M-t Sexps
M-x transpose-paragraphs
M-x transpose-sentences

undo/redo

C-/ Undo
C-? Redo
C-x u Undo-tree-mode if activated !!!

view interfaces

;; buffer view
C-x M-b View buffers (use ibuffer)

;; other buffer use
C-x 4 C-f Find file
C-x 4 C-o Open other buffer
C-x 4 d Dired
C-x 4 b Choose and activate other buffer
C-x 4 0 Kills buffer and window
C-x 4 p Project command !!!

;; window use
C-x 0..3 Delete and split windows
C-x o Switch active window (mapped to M-o)
C-x left|right Cycles buffers
C-c left|right Goes to other configurations with winner-mode

;; other frame use !!!
C-x 5 2 New frame
C-x 5 0 Delete active frame
C-x 5 1 Delete other frames
C-x 5 C-f Find file
C-x 5 d Dired
C-x 5 o Buffer in other frame
C-x 5 p Project in other frame

visit

C-x C-f visit file
C-x C-f C-f visit new buffer (no ido)

windows

C-x 0 deletes active window
C-x 1 deletes other windows (f12 and C-f12)
C-x 2 split window below
C-x 3 split window right
C-x o switch to other window (M-o)
M-x winner-mode remembers window settings
C-x 4 <keys> does things in other window
C-x 5 <keys> frame management

auto-capitalize-mode

cl is deprecated, use cl-lib in the autocapitalize.el file

calc

Start C-x * c; Quick C-x * q
Quit C-x * x | q
Getting Started ‘h i’
Tutorial ‘h t’
Reference ‘h s’; C-x * s
Key, Function, Variable ‘h k’, ‘h f’, ‘h v’
Demo Calc has something for everyone! Many of the functions are with capital letters such as Q sqrt, I inverse, S sine. Keypad mode C-x * k brings up a pad that works with mouse. Grabber C-x * g|r into vector|matrix. Undo U. To transpose use v t, Convert units ’ 43 mi/hr with u c new units. Date arithmetic t N, 7 +- Algebraic eqn (entered with ’ [2x+5=11]) solving requires ‘a S’ then variables to solve for. Help with h k|f whatever and l to get back. To remove, press <DEL> and to exit q.

codeium crane

[This aibot just does not work well yet 250518. It is nice to some extent for working in emacs, but the website seems unable to produce correct code in some situations and has on occasion gone into a repeat loop. There seem to be some neat things coming up like Windsurf, but I’m not sure of what use that will be for emacs.]

Codeium is a free AI software that offers developers code completion, chat, and search capabilities in over 70 languages across 40+ integrated IDEs. We set up codeium.el for Emacs with corfu.el (a completer frontend), cape.el (which provides useful functions for completion), and eglot (the emacs client for language server protocol). The code is modified from the very helpful jeff-phil setup and much of the annotation is provided by Perplexity AI (though edited).

While there is considerable and grateful plagerization from various sources, some of the code has been re-written for my own purposes.

The init-codeium.el page begins with the following requires, before getting into the actual code:

(require 'codeium)
(require 'corfu)
(require 'cape)
(require 'eglot)

Idealings with codeium

Code below sets up various configurations and functions for the Codeium package in Emacs. It enables Codeium for code completion globally, configures the mode line display, sets up filters for API calls, and defines functions to retrieve the document text and cursor offset for better performance. The annotations explain the purpose and functionality of each part of the code.

;; The code below sets up various configurations and functions for the
;; Codeium package in Emacs. It enables Codeium for code completion
;; globally, configures the mode line display, sets up filters for API
;; calls, and defines functions to retrieve the document text and
;; cursor offset for better performance. The annotations explain the
;; purpose and functionality of each part of the code.

;; Add Codeium completion function to completion-at-point-functions
(add-to-list 'completion-at-point-functions
             #'codeium-completion-at-point)
;; Adds the Codeium completion function to the list of functions
;; called when trying to complete text at point, making Codeium
;; available for code completion globally.

;; Do not use popup boxes
(setq use-dialog-box nil)
;; Disables the use of popup boxes, as Codeium likely uses other means
;; of displaying information.

;; Configure Codeium mode line
(setq codeium-mode-line-enable
      (lambda (api)
        (not (memq api '(CancelRequest Heartbeat AcceptCompletion)))))
(add-to-list 'mode-line-format '(:eval (car-safe codeium-mode-line)) t)
;; Configure Codeium to display its status in the mode line, filtering
;; out certain API calls (CancelRequest, Heartbeat, AcceptCompletion)
;; from being displayed. Doing so can be useful for debugging or
;; monitoring purposes.

;; Filter for enabled Codeium API calls
(setq codeium-api-enabled
      (lambda (api)
        (memq api
              '(GetCompletions Heartbeat CancelRequest
                               GetAuthToken RegisterUser
                               auth-redirect AcceptCompletion))))
;; Sets a lambda function to filter for enabled Codeium API calls,
;; allowing only certain API calls (GetCompletions, Heartbeat,
;; CancelRequest, GetAuthToken, RegisterUser, auth-redirect,
;; AcceptCompletion) to be processed.

;; Set document text for better performance
(setq codeium/document/text
      (lambda ()
        (buffer-substring-no-properties
         (max (- (point) 3000) (point-min))
         (min (+ (point) 1000) (point-max)))))
;; Sets a function to retrieve the document text for Codeium,
;; retrieving a portion of the buffer text around the current point
;; (up to 3000 characters before and 1000 characters after) to improve
;; performance.

;; Set cursor offset for better performance
(setq codeium/document/cursor_offset
      (lambda ()
        (codeium-utf8-byte-length
         (buffer-substring-no-properties
          (max (- (point) 3000) (point-min)) (point)))))
;; Sets a function to calculate the cursor offset for Codeium,
;; calculating the byte length of the text from the start of the
;; document text up to the current point, likely used by Codeium to
;; determine the position of the cursor within the document text.

Landing on corfu

The setup here is quite simple since we only enable auto-completion and provide global availability for corfu.

(setq corfu-auto t)                     ;enable auto completion
(global-corfu-mode)                     ;enable corfu-mode globally

Weaving the capestry

The cape package is a completion engine that provides numerous completion commands in various situations and each has its own keybinding. Uncomment any of the items below to enable.

[TODO: We don’t know what a lot of these items do since we adapted the code from the cape.el page.]

;; Bind dedicated completion commands
(global-set-key (kbd "C-c p p") 'completion-at-point) ;; capf
(global-set-key (kbd "C-c p t") 'complete-tag)
(global-set-key (kbd "C-c p d") 'cape-dabbrev)
(global-set-key (kbd "C-c p h") 'cape-history)
(global-set-key (kbd "C-c p f") 'cape-file)
(global-set-key (kbd "C-c p e") 'cape-elisp-block)
(global-set-key (kbd "C-c p k") 'cape-keyword)
;; (global-set-key (kbd "C-c p a") 'cape-abbrev)
;; (global-set-key (kbd "C-c p l") 'cape-line)
;; (global-set-key (kbd "C-c p w") 'cape-dict)
;; (global-set-key (kbd "C-c p :") 'cape-emoji)
;; (global-set-key (kbd "C-c p s") 'cape-elisp-symbol)
;; (global-set-key (kbd "C-c p \\") 'cape-tex)
;; (global-set-key (kbd "C-c p _") 'cape-tex)
;; (global-set-key (kbd "C-c p ^") 'cape-tex)
;; (global-set-key (kbd "C-c p &") 'cape-sgml)
;; (global-set-key (kbd "C-c p r") 'cape-rfc1345)

;; Add to the global default value of `completion-at-point-functions'
;; which is used by `completion-at-point'. The order of the functions
;; matters, the first function returning a result wins.  Note that the
;; list of buffer-local completion functions takes precedence over the
;; global list. However, since we don't really know what most of these
;; are for, we are just commenting them out.
(add-hook 'completion-at-point-functions #'cape-dabbrev)
(add-hook 'completion-at-point-functions #'cape-history)
(add-hook 'completion-at-point-functions #'cape-file)
(add-hook 'completion-at-point-functions #'cape-elisp-block)
(add-hook 'completion-at-point-functions #'cape-keyword)
;; (add-hook 'completion-at-point-functions #'cape-tex)
;; (add-hook 'completion-at-point-functions #'cape-sgml)
;; (add-hook 'completion-at-point-functions #'cape-rfc1345)
;; (add-hook 'completion-at-point-functions #'cape-abbrev)
;; (add-hook 'completion-at-point-functions #'cape-dict)
;; (add-hook 'completion-at-point-functions #'cape-elisp-symbol)
;; (add-hook 'completion-at-point-functions #'cape-line)

One capf to joule them all

The my/capf-for-all alias is used for the cape-capf-super function for better clarity. The code sets the cape-capf-properties are set for codeium and dabbrev (though more can be added).

(defalias 'my/capf-for-all
;; Create an alias 'my/capf-for-all' for the following expression
(cape-capf-super                      ;Combine multiple completion sources using cape-capf-super
 (cape-capf-silent                    ;Suppress the annotation for the following completion sources

  ;; Set properties for the codeium-completion-at-point function
  (cape-capf-properties #'codeium-completion-at-point
                        :annotation-function
                        ;; Define an anonymous function to provide the annotation
                        #'(lambda (_) (propertize "  odeium"
                                                  'face font-lock-comment-face))
                        ;; Set the company-kind property to 'magic'
                        ;; It is a cape property specifying type of completion candidate
                        ;; not really required, but can provide a better visual experience
                        :company-kind (lambda (_) 'magic)))

 ;; Prevent duplicates from the eglot-completion-at-point function
 (cape-capf-buster #'eglot-completion-at-point 'equal)

 ;; Set properties for the cape-dabbrev function
 (cape-capf-properties #'cape-dabbrev
                       :annotation-function
                       ;; Define an anonymous function to provide the annotation
                       #'(lambda (_) (propertize " Dabbrev"
                                                 'face font-lock-comment-face)))

 ;; if line gets too chatty
 (cape-capf-silent
  ;; Suppress the annotation for the following completion source
  (cape-capf-properties #'cape-line
                        ;; Set properties for the cape-line function
                        :annotation-function
                        ;; Define an anonymous function to provide the annotation
                        #'(lambda (_) (propertize " Line"
                                                  'face font-lock-comment-face))))))

Engaging in eglottony

Completion settings for eglot managed modes are configured here pulling everything together to work with eglot.

(defun my/eglotted-modes-config ()
  "Configuration of settings for eglot-managed modes"
  (setq-local completion-category-overrides '((file (styles partial-completion)))
              ;; Set completion-category-overrides for file category
              completion-category-defaults nil
              ;; Clear completion-category-defaults
              completion-styles '(orderless partial-completion basic)
              ;; Set completion styles
              completion-at-point-functions (list #'cape-file 'my/capf-for-all)
              ;; Set completion-at-point-functions to include cape-file and my/capf-for-all
  ))

;; Finally, add the my/eglotted-modes-config function to the
;; eglot-managed-mode-hook so it gets called when eglot runs.
(add-hook 'eglot-managed-mode-hook #'my/eglotted-modes-config)

This setup is mainly to get codeium to work on clojure scripts with eglot as the language server client. When programming elisp though we don’t need capfs and eglot. In fact, the codeium.el page provides code to work with elisp (see cl-letf* function towards the bottom).

[TODO: try to get codeium to work with shell better.]

completers

Several have been examined including ido|fido, vertico, ivy, but none offer as much as helm. Though it may have been slow on old computers, speed is not an issue now.

helm

wiki Doing things in helm is different from vanilla emacs:

use windows rather than minibuffer
use pattern rather than <tab>
<tab>|C-i gives available actions
C-j|C-z persistent action (helm buffer persists)
in a helm buffer, marking is not done with m
embedded help in org buffer with C-h m

Certain emacs commands have been helmed.

ido

Though built-in and mostly nice, the ido-switch-buffer doesn’t always work. The ido flex matching was excellent for the most part, though it could get confusing in some situations (helm’s locate is far better - almost as powerful as recoll file find or fzf fuzzy find.) Though fido is also built-in, we found no real advantage to it.

vertico

It is supposed to be an ido|ivy replacement forked from selectrum, but doesn’t seem to have any real advantages. Easy to get up and running, but not so easy to figure out how to optimize things. One advantage is it works with existing emacs bindings.

ivy

We use to use this as well as icicles many years ago, but switched to ido because things weren’t as messy.

daemon as systemd

$ systemctl –user enable –now emacs cp usr/lib/systemd/user/emacs.service .config/systemd/user systemctl enable –user emacs

default editor

export EDITOR=emacs How to change the default text editor on Linux

dired

Edit directories with emacs which is possibly far more useful than we ever thought. Reference Card pdf.

enter

C-x d from eshell or provide a pattern!

C-x (4|5) d ~/
C-x d ~/*
C-x d ~/.*
C-x d ~/D*
C-x d ~/**/*.el ;globstar should work but doesn't
C-x (4) C-j if done in dired gives parent dir

navigation

n,p
j <==> C-x C-f
M-s f (M-)C-s searches just for filename (or change dired-isearch-filenames to t)

deletion

DEL doesn't work
d,u,x does (on regions too)
m,D is an alternative

multiple flagging

#,~,. probably not going to use
% g flag garbage files .log, .toc, .dvi, .bak ...
% d regexp very nice!!

toggle hidden files

C-u s and -al -> -l or may be this [https://www.emacswiki.org/emacs/DiredOmitMode][Toggle Showing Dot Files]

visiting files

RET|f
o,C-o
v readonly view
^,..

not same as openwith which can be done in various ways, but simplest for now is M-! <app> file.
this isn’t as convenient as nnn, but should do for now. dired+ apparently has a way to do this.

visit dir

use a instead of C-m to discard ancestors use C-x C-j to jump to parent (but that will leave trail), so go to .. and use a

dired marks and flags

these offer many possibilities, but are awkward to remember right now.

m can do region or C-u N as well
* N number and size of files in marked region
* * mark exe files
* @ mark slinks
* s mark all in dir
* / mark dirs
U remove all marks

mark by regexp
% m|g
% d


some of these may be useful though we can't see why right now:\\
* t toggle
M-{} move up/down to next marked file
C-/ undo dired changes
* c old new: * c D t  * c SPC D  * c t SPC

operating on files

D
Z
C new
R new to existing dir (dired-create-destination-dirs)
S new

A regexp very nice!!
Q regexp very nice!!

% u,l upper,lower case filenames

probably not useful at this stage
M modespec
G newgroup
O newowner
W open file externally

transforming filenames

% R,C,H,S computes new name by regexp doing a search-replace not sure how important this is since we can treat the dir as emacs buffer

subdirs

i
C-u C-SPC
$
M-$
C-u k

symlinks

Go to item symlink points to

find and locate

these are done from dired

M-x find-name-dired
M-x locate

view images

C-t d on marked image files
M-x image-dired

rotations possible with jpegtran

delete and tag of image files also possible, so can this replace gthumb!!!

other features

M-x dired-create-empty-file
M-x dired-compare-directories
M-s a C-s isearch on filenames

zz

Dired, the Directory Editor DiredMode

elfeed

Elfeed Emacs Web Feed Reader How to Use Emacs for RSS with Elfeed A workflow for reading, managing and discovering ML research papers with Emacs Note taking 3: Notes from elfeed entries !!!

Setup init-elfeed.el with:

vars for urls (because they can get very long and look messy)
the feeds var elfeed-feeds
the update var elfeed-search-update gets the feeds
delete feed by removing it on M-x customize elfeed-feeds

index mechanism

trying to figure out how to search the feeds from .elfeed in data/1d/1da2ce0dd6436d802f0e1b35ad60f8b745780a5b in index “https://www.sciencedaily.com/rss/top/science.xml” nil) (“www.sciencedaily.com” . “https://www.sciencedaily.com/releases/2021/04/210405150225.htm”) #s(elfeed-entry (“www.sciencedaily.com” . “https://www.sciencedaily.com/releases/2021/04/210405150225.htm”) “Elasticity to position microplates on curved 2D fluids” “https://www.sciencedaily.com/releases/2021/04/210405150225.htm” 1617649345.0 #s(elfeed-ref “1da2ce0dd6436d802f0e1b35ad60f8b745780a5b”) html nil (unread)

There is likely no benefit to recolling the entire .elfeed dir because

we really don’t want all the feeds
those that we do want probably don’t have enough text content
the title of the article contains most of the info and that is already searchable

Saving specific articles to zitems may be useful, so an elisp script does the job mostly written by perplexity!

emailing a buffer

C-ret to setup email C-M-p to prger

setup init-elfeed

use variable for feeds remove titles, but set them as tags (setq aiblog <URL>) though we did have (setq aiblog (list <URL> <TITLE)) title now "" and set as a tag

regex group replacement

(car aiblog) -> aiblog (car $......$) -> \1

eshell

some of these items apply to emacs in general.

builtins

occur <regex> displays matching items in history

cd =, cd =foo, cd -N are excellent for moving into directories!

sudo works using eshell/sudo, but as an alias (eshell-prefer-lisp-functions didn’t work)

built-in variables:
$+, $-, $$, $? (with ending underscores), may be useful for scripting
also, note specific args handling with $*, $1, $2 …

eshell to have $PATH

;; match zshenv PATH (setenv “PATH” (shell-command-to-string “echo -n $PATH”))

that seems a lot simpler (not sure if it will work either), however, we did it another way by

echo $PATH > tmp
edit tmp
PATH=$(cat tmp)

after the above in zsh, we go to eshell and

(setq path <copied from echo $PATH>)
(setq exec-path (split-string path “:”))

expansions

(setq var "this is a string with extension.test")
$var
$'var' disambiguate
$#var length

$(sexp) can be used in a string
${cmd} output of cmd
$<cmd> output to tmpfile

$var[i] ith element of sequence (string split on spc)
$var[: n0 n1 n2 ...] split on : ;interpolation into larger string !!!
$var["\\\\" i] ;!!!
$var["[0-9]+" 10 20] ;!!!
$var[hello] ;!!!
$#var[hello] ;!!!

fix the history!! issue

customize eshell-expand-input-functions by INS eshell-expand-history-references t

for loop

for VAR in TOKENS { command invocation(s) }
for i in (split-string
          (shell-command-to-string "cat /etc/passwd") "\n")
{(print (string-join (list i "BOO") " "))}

help

help on function at point: C-h f

history

M-p|n works as expected

M-r|s is a bit awkward compared to C-r in regular shell, though there may be a better way

unlike C-r in a regular shell, M-r doesn’t keep cycling back … need to figure out how to deal with this

TODO pcomplete

not sure what this is completion concept

redirection

mostly same as zsh, but we also have these neat items

>>> #<buffer-name>
- > /dev/clip

as well as being able to define virtual targets with ("/dev/name" function mode)

scripts

use source from eshell and eshell-source-file from rest of emacs

ssh to remote

C-x C-d /ssh:lentil: M-x eshell and it works perfectly!

sudo issue

which sudo give /usr/bin/sudo - even when lisp functions are prefered !!!!! so we use an alias sudo eshell/sudo $* apparently the documentation is misleading

alias sff find-file /sudo::$1

zz

eshell manual

ess and paredit

these two don’t play nicely and you get c () so remove paredit from .R files and things seem to be just fine

eww

The browser is quite impressive doing what it is primarily designed for - browsing simple sites and reading docs. However, some of the keybindings and commands don’t seem to work. Not an important matter though since we likely won’t use it much.

flys

There is a fly for spelling, flyspell, and 2 for code syntax checking, flymake (builtin to emacs), flycheck (from elpa).

flyspell

Used for spell checking and allows adding to the dictionary.

flymake

Used for checking code syntax and works with LSP servers through clients like lsp-mode and eglot.

flycheck

Does the same as flymake, but presumably works better and with more languages (eg sh-zsh). The checking seems to be automatic, but with lsp I’m not sure if it is required. Quite possibly flymake is sufficient, but we should test both out. Lsp uses flycheck if it is available automatically, so we don’t have to specify that flymake be hooked for a language.

fdm

fetch and deliver mail it is MDA which can fetch https://wiki.archlinux.org/title/Fdm simple config for our notmuch purposes

following symlinks

C-h v vc-follow-symlinks and change the value this will handle git asks when opening a file !!!

fountain-mode

screenplain

doesn’t handle C-c C-a synopsis unless there is a blank line macro fynopsis takes care of this and is bound this way: (add-hook ‘fountain-mode-hook ; enters synopsis for screenplain issue (lambda () (local-set-key (kbd “C-c C-a”) ‘fynopsis)))

greppers

These search for occurrences of pattern in the buffer and/or in files.

emacs grep is good and straightforward to use from the menu requiring only a pattern and a directory of files.

helm ag M-g f will give pattern at|before point by default for the directory, so it can do much the same job as rg, but only on the directory you are already in. It is faster to use, if you want to search pwd only and not some other directory, since you have to go there, whereas C-c s with rg let’s you setup in whatever directory you want to.

rg C-c s is supposed to be faster with better customization. Though deadgrip was fine (and we like the rip pun), it was awkward to get into the directory we wanted. The rg package offers many more features and is into higher versions.

We find rg to be easier to setup and use too - with a guiding menu after C-c s and the docs are quite thorough as well.

To use ripgrep instead of grep see article. At this point, we see no real advantage to replacing grep with ripgrep. The code to insert in init.el is something like this:

(grep-apply-setting
  'grep-find-command
  '("rg -n -H --no-heading -e '' $(git rev-parse --show-toplevel || pwd)" . 27))

Don’t need to do this though because (require ‘rg) (rg-enable-default-bindings)

is quite sufficient to work with rg and also have grep around.

searchers work only in the buffer and include C-s|r and M-x occur which will make a buffer with links for the pattern, so it gives you the same thing that search does, but in a separate buffer and with every occurrence rather than forward C-s or reverse C-r.

ibuffer

Get in with C-x M-b

Delete multiple buffers quickly

% n|m <> RET
D y

images

Insert an image with

#+CAPTION: This is the caption for the next figure link (or table)
#+NAME:   fig:SED-HR4049
[[./img/a.jpg]]

junk deposit issue

emacs -nw producing [Ihello] and other junk on first line

this is happening because of the line in init-modes.el

(sp-pair "[" "]" :wrap "M-[")

other braces {} () don’t seem to have any issues

key (un)bind

(global-set-key (kbd "C-M-.") 'eshell)
(global-unset-key (kbd "C-M-."))

lilypond

qpdfview

open the pdf update in lilypond, then refresh with F5

xpdf -open

this works quite well until it is unable for some reason to create a remote socket. however, the problem can be nullified by: rm /tmp/default which removes the socket process (i guess?) unfortunately, when operating as a server, the textedit:// feature does not work as effectively though you still get the line number of where you want to go to in the .ly file.

midi

Lilypond-midi-player variable set to timidity|fluidsynth

lsp

Language Server Protocol Support for Emacs Emacs and the Language Server Protocol

notmuch

the idea will be to use search/tag rather than folders to get things done

setup

%notmuch setup -> ~/.notmuch-config used to have .emacs.d/notmuch-config.el with mainly the Q binding, but that is now in init-email.el

starting

emacs -e “notmuch” with client it used to be different: emacsclient -e “(elfeed)” -e “(notmuch)” & emacsclient -c ~/aa/aa.org &

various commands

search|show|address|count reply tag dump|restore config

html

(global-set-key (kbd “<f6>”) ‘org-mime-edit-mail-in-org-mode) use to open a second window and write in orgmode C-c C-c will update orgified buffer to the original

(global-set-key (kbd “M-<f6>”) ‘org-mime-htmlize) htmlize the orgified buffer before sending with C-c C-c (as before)

searching

wildcards using ‘*’ operators +|-|AND|NOT|XOR|OR| brackets proximity NEAR ADJ phrases “Debian Project” <=> Debian ADJ/1 Project prefix freeform idea, but uses full item eg from: instead of f: range date:yr-mo-da..yr-mo-da | -5da..now | yesterday

man notmuch-search-terms

fetch/trash mechanism

use nmprg.sh) f|c|p|t

f files are left on server but not duplicated c) fdm fetch will remove them from server p) polls t) trash require ‘S’ on emails then nmprg.sh t moves them to ~/.trash

be careful using c) because if you read using spark or whatever on the server and leave the item as read, then using this removes them from the server, but we’re not sure where the email actually goes! yes we are now!!

we must be sure that we are running nmp from ~/ otherwise mail/ will be created wherever we are!

autocomplete email addresses

chose to use bbdb and use bbdb-vcard with contacts.vcf exported from google Go to http://contacts.google.com, click the menu “Export…>>vCard format (blah, blah …)”. Click “Export” button. Download the contacts.vcf into ~/Downloads then load into an emacs buffer and run

bbdb-snarf-vcard-buffer

Remember to save the bbdb buffer to .emacs.d/bbdb

gnus-alias to select identity

https://notmuchmail.org/emacstips/#index16h2 didn’t set rules use M-x gnus-alias-select-identity

external handlers for attachments

w - to save the attachment - though the default doesn’t always seem to work. . o allows reading the item provided you specify the type

sending email

evolution removals

for some reason, the setup for smtp wasn’t being picked up in the init.el file and that is why evolution kept coming up. instead of smtpmail-send-it the setting had changed to mailclient-send-it. this is nice to know in case we should ever require it in the future.

;; smtp (setq smtpmail-default-smtp-server “shawmail.gv.shawcable.net” send-mail-function ‘smtpmail-send-it user-full-name “prad” smtpmail-local-domain “towardsfreedom.com” user-mail-address “prad@towardsfreedom.com”) (require ‘smtpmail)

otherwise it defaults to evolution because gnome sets it, though we can change it to notmuch through settings.

vivaldi mail issue

however, what is the problem replying to vivaldi? no issues sending to vivaldi or replying to any other place it seems. changing the smtp to “mail.towardsfreedom.com” sent, but still the same problem with replying to vivaldi!

<2021-06-21 Mon> we are sending pradagio@vivaldi.net emails through smtp.vivaldi.net because shawmail wouldn’t send (even though it appeared to do so).

2 items noted:

gnus alias setup is sufficient (no need for msmtp) (“tf” nil ;; Does not refer to any other identity “prad <prad@towardsfreedom.com>” ;; Sender address “towardsfreedom.com” ;; Organization header ((“X-Message-SMTP-Method” . “smtp smtp.towardsfreedom.com 587”)) nil ;; extra body text “/.signature”) (“fm” ;;formal tf nil “prad basu <prad@towardsfreedom.com>” “prad basu” ;; Organization header ((“X-Message-SMTP-Method” . “smtp smtp.towardsfreedom.com 587”)) nil ;; No extra body text “/.signature.fm”) (“vv” nil “prad <pradagio@vivaldi.net>” “prader” ;; Organization header ((“X-Message-SMTP-Method” . “smtp smtp.vivaldi.net 587”)) nil ;; No extra body text “~/.signature.vv”) )
mail-host-address (setq mail-host-address “localhost”) couldn’t figure out how to change it via smtp on gnus-alias, but that’s probably not important however, we need this to prevent Message ID <87lf72na5y.fsf@schnell.i-did-not-set–mail-host-address–so-tickle-me> doing (“Message-ID” . “towardsfreedom.com”) didn’t work though it did get the item in there. receiving end produced: Message ID <60d15486.1c69fb81.90dc4.e612SMTPIN_ADDEDBROKEN@mx.google.com>

still getting a spf softfail: send from vivaldi -> tf reply from tf -> vivaldi don’t know why, but it probably doesn’t matter.

also unclear why .authinfo required vivaldi smtp info but nothing for tf

orgmode

manual

org macros

github link The macros in Org allow you to add text or code to an Org document that will be expanded during export: the text between the 3 pairs of accolades is expanded so that readers can see it in the exported versions of the document.

Some people (such as me) use it to construct complex HTML or LaTeX code.

13.5 Macro Replacement

the .#file issue with org-agenda-files

When that variable is set to ~/aa/*.org various files within the aa directory kept appearing making cycling awkward. We haven’t found out why though the problem started around <2023-07-22 Sat> for some reason. Possibly, having the default ~/org directory would have caused those files to end up there (which really isn’t the best solution). Setting the default org-directory to aa seems to have worked albeit the idea is pretty counter-intuitive.

org agenda

Cycle the org agenda directory with C-’ (which seems to be a default setup). packages

melpa vs melpa-stable

Packages are identified when we look at M-x package-list-packages

update packages

M-x list-packages and it will automatically refresh the archive contents.

Then, press U to mark all upgradable packages to be upgraded.

Last, press x to perform the new updates.

Emacs will then download and install all upgradable packages. It will also ask you whether to delete the old packages: M-x package-autoremove

Doing it this way seems better than package-refresh-contents package-install (if package-reinstall doesn’t work)

table

If ; isn’t a separator by default: C-u C-u C-u C-c | ; RET

package

removing packages

To remove unnecessary or unwanted packages in Emacs, follow these steps:

Open Emacs.
Press `M-x` (Alt + x) to invoke the command prompt.
Type `package-list-packages` and press Enter.
In the Packages buffer, locate the packages you want to remove.
Mark them for deletion by pressing `d` (package-menu-mark-delete).
Execute the deletion by pressing `x` (package-menu-execute).
Confirm the removal when prompted in the mini-buffer.

This process will remove the actual files associated with the packages from the `~/.emacs.d/elpa` directory¹. Happy cleaning! 🧹🚀

Source: Conversation with Copilot, 6/18/2024 (1) How to remove an installed package in Emacs (ver 24+)?. https://stackoverflow.com/questions/16469600/how-to-remove-an-installed-package-in-emacs-ver-24. (2) uninstallation - How remove emacs full? - Stack Overflow. https://stackoverflow.com/questions/57895073/how-remove-emacs-full. (3) Package Menu (GNU Emacs Manual). https://www.gnu.org/software/emacs/manual/html_node/emacs/Package-Menu.html. (4) apt - how to uninstall emacs from command line? - Ask Ubuntu. https://askubuntu.com/questions/604733/how-to-uninstall-emacs-from-command-line. (5) Uninstalling a package and all of its dependencies - Emacs Stack Exchange. https://emacs.stackexchange.com/questions/31888/uninstalling-a-package-and-all-of-its-dependencies.

update packages

Certainly! To update packages in Emacs, follow these steps:

Open Emacs.
Type `M-x list-packages` and press Enter. This command lists all installed packages and automatically refreshes the package archive contents.
In the Packages buffer, press `U` to mark all upgradable packages.
Finally, press `x` to perform the updates. Emacs will download and install the new versions of the packages. It will also prompt you to delete the old packages if necessary¹².

Happy updating! 🚀🔧

Source: Conversation with Copilot, 6/18/2024 (1) elpa - Updating packages in Emacs - Stack Overflow. https://stackoverflow.com/questions/14836958/updating-packages-in-emacs. (2) How to upgrade packages installed by ELPA in Emacs?. https://www.systutorials.com/how-to-upgrade-packages-installed-by-elpa-in-emacs/. (3) Keeping Your Emacs Packages Up to Date - System Crafters. https://systemcrafters.net/emacs-from-scratch/keeping-your-packages-up-to-date/. (4) How to update packages installed with use-package. https://emacs.stackexchange.com/questions/31872/how-to-update-packages-installed-with-use-package. (5) undefined. http://tromey.com/elpa/. (6) undefined. http://elpa.gnu.org/packages/. (7) undefined. http://marmalade-repo.org/packages/. (8) undefined. http://melpa.milkbox.net/packages/.

profiler

Option 2: The profiler

Invoke M-x profiler-start RET RET (the second RET is to confirm cpu);
Do some typing, preferably an entire paragraph or more;
Invoke M-x profiler-report.

That will give you a buffer describing the cpu time taken by each function. Hitting TAB on a line expands it to display the functions inside it. Navigate this buffer until you find out which function is taking so much CPU time. https://emacs.stackexchange.com/questions/5359/how-can-i-troubleshoot-a-very-slow-emacs

we tried this to figure out AC mode was slowing things down, but we already knew that. Not sure what to do about it though.

pinky solution

One way of not injuring the left pinky even after making CAPS into CTRL is to use different fingers to push it. That way the usage is distributed and the hands move around more too.

TODO regexp

Likely not worth paying in-depth attention to since it is not pcre. However, some of the uses can be helpful especially for simple things.

search file system

SFS is a collection of inter-operating emacs tools to make accessing and organizing your data faster and more ergonomic. It is largely powered by the Recoll file-indexing tool.

This may be worth pursuing eventually !!!

sessions

Use the default M-x save-sessions to ~/.emacs.d/session-buffers to get all back. Sometimes certain buffers get persistent because session-buffers keeps them, so delete the buffers and save-session again.

smartparens

Pair management haven’t been able to deal with ~~ and // in org-mode yet !!!

spellcheck

be sure aspell or whatever is installed. M-$
Check and correct spelling of the word at point (ispell-word). If the region is active, do it for all words in the region instead.

M-x ispell
Check and correct spelling of all words in the buffer. If the region is active, do it for all words in the region instead.

M-x ispell-buffer
Check and correct spelling in the buffer.

M-x ispell-region
Check and correct spelling in the region.

tab-bar

Likely not of much use since we tend to prefer buffers anyway.

C-x t 2 creates a new tab
C-tab next tab
C-S-tab prev tab
C-x t r rename tab
C-x t m move tab to right
C-x t RET select tab by name
M-x tab-list list tabs

tab completion

Setup in init-keybindings.el with (require ‘helm) (define-key helm-map (kbd “TAB”) ‘helm-execute-persistent-action)

terminals

The ansi-term gives colors, but term doesn’t. C-c C-j linemode to use M-x

Code runs ansi-term without the zsh prompt:

;; use ansi-term with zsh
(defun zterm ()
  "term to use zsh"
  (interactive)
  (ansi-term "zsh"))

(M-)s-a (close) open terminal in second window

;; other terms
s-' ansi-term full window
s-. eshell
s-y term with zsh (likely to use very rarely)

There is nothing wrong in allowing ansi-terms to accumulate even if we do not go back to them, though we might if they are involved in specific tasks. In order to name the ansi-terms we need to go into line-mode (C-c C-j) and not character-mode (C-c C-k), so we can use M-x rename-buffer.

In line mode, it reads an entire line and permits various emacs editing operations on it, then sends it to the process all at once when you’re done (press Enter). In char mode, each character is sent through as soon as you press the key - dealing with lines, editing, and so on is up to the process running in the terminal, such as your shell.

Why might you want to use one over the other? Line mode allows you to perform Emacs editing operations on what you write, to accumulate multiple lines to send through at once, and to perform tab completion of filenames in Emacs. See the list of commands in shell mode for details. Character mode lets you use all the features of the underlying process, like its own (likely better) tab completion, rather than the Emacs operations, and also to send through inputs that Emacs would have interpreted as referring to its own commands (like Ctrl-X). This is also the mode you’d want if you wanted to run vim inside emacs.

An ordinary terminal operates in the equivalent of character mode. Line-mode vs Character-mode explanation

Clean-up of ansi-terms can be done in the same way as clean-up of sent-mail buffers: % n RET <pattern> marks the buffers by name after which use D to delete them. See .

Macros have been created for ‘aterm s-a ‘aterm-rm M-s-a

For some reason, aterm does not let M-x to work - get execute: showing up in the window. Usually we would M-x to rename-buffer, but a shortcut exists: C-x x r and that will bring up the rename-buffer command.

themes

(add-to-list ‘custom-theme-load-path “~/.emacs.d/themes/”) M-x customize-themes

tramp

We can use ssh|sudo from rootdir with // and once we tramp into another computer the eshell works there (should we want it to).

hanging with ssh

https://www.emacswiki.org/emacs/TrampMode put in the remote .zshrc

[[ $TERM == "dumb"]] && unsetopt zle && PS1='$ ' &&

this is a zsh issue because people have said that when the switch to bash it all works.

<2023-09-22 Fri> stopped working for some reason, but a better solution is to use sshx which uses /bin/sh instead of zsh

ssh with sudo

C-x C-f /ssh:psinom::sudo:psinom:

variables

shell-mode-map is void

shell-mode-map is a variable that is defined in the file shell.el which is not loaded yet, so the variable is undefined.

(require ‘shell) before you try use the variable. post

save and apply

This can be done through the emacs customize menu, but to do alter a variable permanently through the minibuffer, use M-x customize-

vc

A “version control system” is a program that can record multiple versions of a source file, storing information such as the creation time of each version, who made it, and a description of what was changed. Emacs VC supports git and several others and is enabled whenever a file under VC is visited.

Commands reachable with prefix C-x v <tab>

Much more to investigate !!!

zz

agenda cycling extra buffers

Cycling through the agenda dir sometimes produces empty buffers of the same name starting with .# and we think this may be because extra files of the same name somehow get created. Deleting the buffers doesn’t get rid of them, because they keep coming back. However, if we M-x reload-dot-emacs after deleting them, the problem goes away.

issue with [I ] on first word

we don’t really know what the problem is though it seems to have something to do with the TERM variable. it occurs with emacs -nw, but not with sudo emacs -nw probably because at the time this was written, there was no .emacs in /root as evidenced by a solution which works with emacs -nw -Q.

What does work if using gnome-terminal is TERM=gnome-256color

enter giving Wrong type argument: stringp, nil

inserting the doc with C-x C-i didn’t work hopefully copying the items one by one will we will also leave orgmode to its own file

close help window without going to it

(setq help-window-select t) that way when using q, it closes the help window since it is already there.

re-able window

C-z isn’t nil (the keybind is commented). It seems to sort of disable the window, but going to another emacsclient window re-activates the cursor. With a standalone emacs, clicking on the menu brings it back.

read-only toggle

C-x C-q

block commenting trick

we don’t really need this since emacs is setup with C-M-c and C-M-S-c for comments

: '
This is a
very neat comment
in bash
'
echo "This is outside the commented block"

Executing the above code would result in: This is outside the commented block https://stackoverflow.com/questions/947897/block-comments-in-a-shell-script

uses

coding
- clj
- zsh
- php
- openscad
- html
email
writing
- stories
- librettos
- papers
lilypond
orgmode

Shell

Tue, 08 Feb 2022 22:18:26 -0800

We use the z shell and program in zsh. All items in .zsh.d with .zshenv and .zshrc symlinked. This setup is easily migrated to other machines. The items below illustrate various shell commands some which are specific to zsh.

adding ~/bin to PATH

### PATH ### typeset -U path #converts to uppercase (to avoid accidental duplication as per bing) path+=~/bin

alternate expressions

ls *glob*.(txt|h) #note use of ()
# *glob*.txt and *glob*.h

block commenting

Though it is unlikely we will actually use this technique, it is handy to know.

echo "this is a block comment" 
: '
    what is this
    actually
    '

capitalizations

All cap specific words in a file using perl|sed. Note that single quotes may work with sed, but not produce desired result with perl.

# perl
perl -pi -e "s/(Eliphas|Retha|Chastity)/\U$1/g" <filename>

for n in  Chastity Retha Eliphas; do # or using loop
    perl -i -pe "s/$n/\U$n/g" <filename>
done

# sed
for n in Cytheri Vaishob Chastity Retha Eliphas; do
        sed -i "s/$n/${(U)n}/g" <filename>
done

compinstall

What does zstyle :compinstall filename do?

It’s for when you run compinstall again. It lets compinstall find where it has written out zstyle statements for you last time. This way, you can run compinstall again to update them. Read in source code.

concat csv

Here are some ways to concat text files.

ls -1 *.csv | while read filnam ; do cat "$filnam" >> output.csv.file; done
ls -1 *.csv | sort | while read filnam ; do cat "$filnam" >> output.csv.file; done
ls -1t *.csv | while read filnam ; do cat "$filnam" >> output.csv.file; done

config files

.zshrc - Should be used for the shell configuration and for executing commands.
.zshenv - Should only contain user’s environment variables.
.zprofile - Can be used to execute commands just after logging in.
.zlogin - Can be used as .zprofile, but read just after .zshrc.
.zlogout - Can be used to execute commands when a shell exit.

cp

replace softlinks with actual file

cp --remove-destination $(readlink <filename>) <filename>

for f in $(find -type l);do # or do all softlinks
    cp --remove-destination $(readlink $f) $f;
done

direct ls output to array

arr=($(ls))

# parsing ls is not a good idea usually so
arr=(*)

# if there are subdirectories
shopt -s nullglob
array=(*/)
shopt -u nullglob # Turn off nullglob to make sure it doesn't interfere with anything later
echo "${array[@]}"  # Note double-quotes to avoid extra parsing of funny characters in filenames

See How do I assign ls to array discussion.

docs for zmv, zcp, zln:

docs packit book scripting

this book contains a lot of useful stuff

zle (ch3)

Emacs users will find themselves at home with the Esc + X sequence. we can use where-is to find an alternative shortcut man zshzle has lots!

globbing (ch4)

the negations ^,~ and alternates (|),[] need work

tilde echo b*~~*.o match starting with b, don’t match anything with o extension echo a*~~*9.* match starting with a, don’t match anything with 9 before the . echo a**9?.* match starting with a, don’t match anything with 9 one item before . echo */.shtmp/* exclude anything within tmp/

qualifiers cheatsheet *(…) TAB *(@) symlinks *(On) reverse sort by name *(.) files only *(@,/) symlinks or directories

TODO things to do

zcalc zargs man zshexpn

dot z files

From posting:

.zshenv is always sourced. It often contains exported variables that should be available to other programs. For example, $PATH, $EDITOR, and $PAGER are often set in .zshenv. Also, you can set $ZDOTDIR in .zshenv to specify an alternative location for the rest of your zsh configuration.
.zprofile is for login shells. It is basically the same as .zlogin except that it’s sourced before .zshrc whereas .zlogin is sourced after .zshrc. According to the zsh documentation, “.zprofile is meant as an alternative to .zlogin for ksh fans; the two are not intended to be used together, although this could certainly be done if desired.”
.zshrc is for interactive shells. You set options for the interactive shell there with the setopt and unsetopt commands. You can also load shell modules, set your history options, change your prompt, set up zle and completion, et cetera. You also set any variables that are only used in the interactive shell (e.g. $LS_COLORS).
.zlogin is for login shells. It is sourced on the start of a login shell but after .zshrc, if the shell is also interactive. This file is often used to start X using startx. Some systems start X on boot, so this file is not always very useful.
.zlogout is sometimes used to clear and reset the terminal. It is called when exiting, not when opening.

Also, the configuration files of random Github users gives possibilities.

expanding and sorting

the o,O sort doesn’t work for ls unless we disable ls sort with -U

Expanding Depending on a Predicate these things may be quite powerful if we learn what to do with them e: *<cmd>

expansions

Expansions Depending On the Permissions might be useful ls *(f755) would list only directories (usually) though ls -d */ simpler however, there are many other features for this

Expansions Depending on Date or Size ls **/*(L0) list all empty files ls **/*(LG+1) list all files >1G ls **/*(mh-1) list all files modified within last hour

find fd

copying files

find MMMM/local/.ocs.agio.organ/cur/ -maxdepth 1 -type f | xargs cp -t mail/cur/
find . -maxdepth 1 -type f |head -1000| xargs cp -t "$destdir"

list by size

find . -size +1M (greater than 1M)
find . -size -1M (less than 1M)
find . -size +1M -exec ls -lh {} +

find only .git dirs

fd –exclude=".cache" –type d –hidden ‘.git$’ find ocs/ -type d -name “.git”

ignore dir

fd –exclude dir find . $-name ".cache" -prune$ -o -name “.git” -type d

modifying files

find . -type d -name "*UNUSED" |
while read f;
  do
    mv $f "${f%/*}/unused";
  done

using for with find is not a good idea for some reason

links

fd . bin/ -t l and broken links are highlighted find bin/ -xtype l just shows the broken links

renumber files

best done with zmv, but using find is still interesting

# filenumberer.sh
k=0
find . -type f |
    while read f;
    do
        k=$(($k+1))
        cp $f ../NEWDIR/${(l(5)(0))k}.pdf
    done
exit

# can be done as per code below, but zmv is preferable

for z in $(find /home/pradocs -name zotes.org); do 
echo `mv $z ${z%/*}/zzotes.org`; 
done

find exec

this is a strange syntax but worth learning.

# find all *.org* in the fireworks/ and replaces localhost:1313 with tf.com
find . -name "*.org" -exec perl -pi -e 's/localhost:1313/towardsfreedom.com/g' {} \;

# remove broken links in bin/
find bin/ -xtype l -exec rm {} \;

# get word count for pdf files
find . -name "*.pdf" -exec wc {} \;

# same as above but line it up nicely
find . -name "*.pdf" -exec wc {} \+

# same as above
find . -name "*.pdf" -print0 | xargs -0 wc
# print0 and xargs -0 handles spaces in filename which is bad idea anyway

exec Really Means exec!
“It uses Linux’s built-in exec to run the command, replacing the current process—your shell—with the command. So the command that is launched isn’t running in a shell at all.” which is why exec ls wipes out the shell!

chaining (instead of piping?) example find . -name “*.page” -type f -exec bash -c “basename “{}” && words-only “{}”” \

ffmpeg

split video files

ffmpeg -ss 00:03:04.560 -i input.mp4 -c copy -t 00:03:39.299 output.mp4

to split video files which sometimes even losscut cannot but ffmpeg can

file into variable

var=$(<stuff.txt)

for itms out of a file

for l in "${(@f)"$(<filename)"}"
         {
             fnumber=$(youtube-dl -F $l | tail -n 1 | cut -f 1 -d ' ')
             youtube-dl --format $fnumber $l
         }

forloops

for i in {1..9}; do echo $i; done for ((i=1; i<10; i++)); echo $i

grep

lines not beginning with #

https://unix.stackexchange.com/questions/60994/how-to-grep-lines-which-does-not-begin-with-or

grep "^[^#;]" smb.conf

In other words, it reports lines that start with any character other than and ;. It’s not the same as reporting the lines that don’t start with and ; (for which you’d use grep -v ‘^[#;]’) in that it also excludes empty lines

heroic commands in linux

https://www.youtube.com/watch?v=Zuwa8zlfXSY

exit terminal but leave all processes running

disown -a && exit

intercept stdout and log to file

cat contacts.org | tee -a log.txt | cat > /dev/null

keep command out of history

start any command with a leading space and it will not go into history

put something in the background** ramdisk for fast rw

mkdir /mnt/ram
mount -t tmpfs tmpfs /mnt/ram -o size=8192M
dd if=/dev/zero of=test.iso bs=1M count=8000

on an ssd the above worked at 2.2G/s, but with ramdisk it was 3.9G/s!

recursive directory creation

mkdir -p folder/{sub1,sub2}/{sub1,sub2,sub3}
mkdir -p folder/{1..100}/{1..10}

redo last command via sudo

sudo !!

particularly helpful if you don’t want to retype a long command.

tunneling

tunneling to connect to cloud through a port of choice:
ssh -L localport:host:port-relative-to-machine login-credentials

we did this at one point to connect to lentil via wifi somehow.

use an editor to alter a long messed up command

push enter on the messed up command and then type fc to work with the command in default editor

execute the command then use bg

largest files and folders identification

du -h | sort -hr | head -n 10
du -ah | sort -hr | head -n 10

ls

inverse

the inverse is giving by ^

ls *M.mp4
ls ^*M.mp4

list only directories

List only directories (post) ls -d – *() ls -ld – *()

new zsh script

cp ~/sh/zztmpls/zshtmpl.sh ~/new.sh

enter title that is descriptive
set VDATe: v1 - M-x org-time-stamp write PURPose
provide desc

block comment above

fill out help section “use” and “how”

add variables and coding

numeric sequence generation

touch a{01..99}.txt works touch a[01-99].txt doesn’t work

x a{2..4}* works x a[2-4]* works

pad with zero

seq -w 1 11

or for more control

for i in {2..11};
do
    echo $u1${(l:2::0:)i}$u2$i
done

and can get something like this

https://tubitv.com/tv-shows/524508/s01-e02-episode-2
https://tubitv.com/tv-shows/524508/s01-e03-episode-3
https://tubitv.com/tv-shows/524508/s01-e04-episode-4

perl onliners

add a line after match

perl -pi -e '/paper.ily/ and $_.="\\include \"../../zztmpls/functions.scm\"\n"' score.ly

prompts

add these to .zshrc:

autoload -Uz promptinit
promptinit
prompt bart (or whatever)

5 prompts:

$PS1
$RPS1 rightside
$PS2 waiting for more input
$PS3 making choices
$PS4 debugging scripts

redo softlinks

find ~/bin -type l -ls | tr -s " " "\t" | cut -f11,13

couldn’t do tr with sed for some reason

https://unix.stackexchange.com/questions/188057/update-symlinks-by-scripting\ though the ideas using realpath will work, it doesn’t do what we are trying to accomplish which is to replace ‘ba’ with ‘sh’ in the softlink

remove empty directories

rmdir --verbose **/*(/^F)
rmdir --verbose **/*(/od)

https://www.zsh.org/mla/users/2005/msg01375.html

remove empty directories recursively

rmdir --verbose **/*(/^F)
rmdir --verbose **/*(/od)

https://www.zsh.org/mla/users/2005/msg01375.html

rename

rename -v pradagio pradesigner */*/* #like grep -r

rsync

exclude

rsync -av --delete --exclude={'dir1','dir2','file1.txt'} ~/ocs ~/mnts/sd64/

run-help

Use M-h on a command

Setup properly in .zshrc unalias run-help 2>/dev/null autoload -U run-help

sed

backref peculiarities

schnell% echo $v
[Integrating Values and Ethics into Wildlife Policy and Management Lessons from North America](http://www.mdpi.com/2076-2615/1/1/126)

schnell% echo $v | sed -r -n 's/\[(.+)\]/\1/p'
Integrating Values and Ethics into Wildlife Policy and Management Lessons from North America(http://www.mdpi.com/2076-2615/1/1/126)

schnell% echo $v | sed -rn 's/\[(.+)\]\((.*)\)/\2\1/p'
http://www.mdpi.com/2076-2615/1/1/126Integrating Values and Ethics into Wildlife Policy and Management Lessons from North America

why does the entire $v get printed in the second item?

getting pics from html file to proper directories

from html source:

<p class="hdr">Iwrters</p>
<p class="dsc">Icquaint yourself with those who contributed to this site over the years!</p>
<br/>
<div style="background-color:#ccccff; margin:1em 1em 5em 1em" >
<img class="floatrig" src="y-283.jpg" width="75" alt="Alexandra"/>
<p class="secitmtle">Alexandra</p>
<p class="secitmdsc">I love animals more than anything else!</p>
<p class="secitmurl"><a href="http://www.30bananasaday.com/profile/AlexandraR?xg_source=profiles_memberList">see</a></p>
<p class="whern"></p>
</div>

<div style="background-color:#ddddff; margin:1em 1em 5em 1em" >
<img class="floatrig" src="y-288.jpg" width="75" alt="Angelkate"/>
<p class="secitmtle">Angelkate</p>
<p class="secitmdsc">Here we find divinity, for we are it.</p>
<p class="secitmurl"><a href="288.html">see</a></p>
<p class="whern"></p>
</div>

need to pull out the y-*.jpg and put it into the alt=“NAME” directory.

the problem couldn’t be solved with sed because non-greedy finds aren’t possible, but cut worked nicely:

sed -n "/^<img class/p" $tf/2.html | cut -d \" -f 4,8 | sed 's/"/:/' > TMP
# need to change the " to : so it can be used as a delimiter for splitting.

while read li; do
    a=("${(@s/:/)li}");
    cp $a[1] ~/hugos/towardsfreedom/content/iwrters/$a[2]/00.jpg;
done < ~/hugos/towardsfreedom/TMP

md to org links conversion** multiple in-place replace:

sed -i
    -e 's/\\skip1//g'
    -e 's/\"//g' 
    -e 's/  / /g' 
    -e 's/\\dynamicUp//g' 
    -e 's/\\set ignoreMelismata = ##t//g' *.ly

replace string with contents of file

sed -e "/Acss/r qq" -e "/Acss/d" index.php

puts contents of file qq right after Acss line and deletes the Acss line people have wondered how to just substitute the string and not the line

swap lines

this sort of does it but not too well there seems to be no way to delete line only the first occurance of a match

sed -e '/subsubtitle/ {h; n;p; x}'  Ta-Ta-Ta.ly | 
sed -e '6d' | 
head

various extraction methods

https://stackoverflow.com/questions/17511639/sed-print-only-matching-group

sed -rn ’s/

(.+)

$(.*)$/ /p’

setting and using an array in for loop

this is a bit weird, but it works

#setup L1 directories
L1dirs=(Iwrter Issays Ithngs Ibrary Ictvty Inotes)
for d in "${L1dirs[@]}"; do
    echo $d
done

though just this will work too:

for d in Iwrter Issays Ithngs Ibrary Ictvty Inotes; do
    echo $d
done

shell options

zsh -l #
zsh -v #
zsh -x #xtrace—which can prove invaluable when debugging your scripts
zsh -f #start a clean instance of zsh using the default settings can set after shell started

shortcut patterns

A-Za-z <==> :alpha: ascii,lower,upper,digit,alnum,print,blank(space,tab),space,punct

<-> numeric x a<70-90>.txt

sh_wordsplit option (bing)

The zsh sh_wordsplit option is a compatibility feature that makes zsh behave like other shells when it comes to word splitting. Word splitting is the process of breaking a string into words based on some delimiter characters, such as spaces or colons. This is often done when passing arguments to a command or looping over a list of values.

By default, zsh does not perform word splitting on unquoted parameter expansions. This means that if you have a variable that contains multiple words separated by spaces, such as `str=“foo bar baz”`, and you use it as an argument to a command, such as `echo $str`, zsh will treat it as a single word and pass it as one argument to echo. This is different from other shells, such as bash or ksh, which would split the variable into three words and pass them as three arguments to echo.

The sh_wordsplit option makes zsh emulate the behavior of other shells and perform word splitting on unquoted parameter expansions. This means that if you set this option, `echo $str will print foo bar baz on three separate lines, as if you had written echo "$ { (@s: :)str}"`. The option also affects other forms of parameter expansion, such as command substitution and arithmetic expansion.

The sh_wordsplit option is useful when you want to write scripts that are compatible with other shells, or when you want to use variables that contain lists of values separated by some delimiter. However, it can also introduce unexpected behavior and errors if you are not careful with quoting. For example, if you have a variable that contains spaces in its values, such as `str=“foo ‘bar baz’ qux”`, and you use it in a for loop, such as `for x in $str; do echo $x; done`, zsh will split the variable into four words and print them on four lines, even though you might have intended to treat `bar baz` as a single word.

A better way to handle lists of values in zsh is to use arrays, which are variables that can store multiple values without relying on delimiters. You can create an array by enclosing a list of words in parentheses, such as `arr=(foo ‘bar baz’ qux)`, and access its elements by using the subscript notation, such as `echo $arr[2]`. You can also use the `typeset -T` builtin command to tie an array variable to a scalar variable, so that any changes to one are reflected in the other. For example, `typeset -T str str_array` will create an array variable `str_array` that is tied to the scalar variable `str`. If you assign a value to `str`, such as `str=“foo:bar:baz”`, the array variable will be updated accordingly, and vice versa.

For more information about word splitting and arrays in zsh, you can refer to the following sources:

[What is word splitting? Why is it important in shell programming?](¹^)
[How to split a string by ‘:’ character in bash/zsh?](²^)
[Z-Shell Frequently-Asked Questions](³^)
[Re: Word splitting with zsh fix](⁴^)

Source: Conversation with Bing, 9/12/2023 (1) zsh - What is word splitting? Why is it important in shell programming …. https://unix.stackexchange.com/questions/26661/what-is-word-splitting-why-is-it-important-in-shell-programming. (2) How to split a string by ‘:’ character in bash/zsh?. https://unix.stackexchange.com/questions/614299/how-to-split-a-string-by-character-in-bash-zsh. (3) Z-Shell Frequently-Asked Questions - SourceForge. https://zsh.sourceforge.io/FAQ/zshfaq03.html. (4) Re: Word splitting with zsh fix. https://www.zsh.org/mla/workers/2006/msg00139.html.

specific computers

# broot setup for specific computers
if [[$HOST = 'schnell' || $HOST = 'adagio']];
   then
       #code
fi

sshfs

paru -S sshfs sshfs <computer>: /home/mnts/<computer> umount /home/mnts/<computer>

tee

reads from the standard input and writes to both standard output and one or more files at the same time (it helped to solve the rejects.sh issue we were having)

it can be used to write to multiple files as well as append:

command | tee file1.out file2.out file3.out
command | tee -a file.out

very useful when combined with sudo:

echo "newline" | sudo tee -a /etc/file.conf

or remotely:

for e in $@; do
    echo "$e REJECT" | ssh REMOTE sudo tee -a /etc/postfix/access
done

text file into array

#!/bin/zsh

# urls should not be separated by blank lines or
# have quotes (it seems or we get missing scheme error)
# getting the urls into an array needs to be looked at

ln=1
for l in "${(@f)"$(<./urls)"}"
         {
             fn=$ln.mp4
             #echo "wget -O $fn $l"
             wget -O "$fn" "$l"
             ((ln++))
         }

echo "all done"

exit

variables as commands

setopt sh_wordsplit otherwise $s=“sleep 2” is non-functional or $e=“emacsclient -c” is not found

zmv

We should use this since it is zsh perl-rename may be better in some situations rename is rather limited except for very straightforward stuff

Very powerful commands though the syntax may be difficult to remember unless we get used to it through regular practice which is unlikely since we don’t do this too often. So looking it up is better.

This is a multiple move based on zsh pattern matching. To get the full power of it, you need a postgraduate degree in zsh. However, simple tasks work OK, so if that’s all you need, here are some basic examples:

foo.txt to foo.lis, etc

The items below shows move, copy, and link using zmv.

zmv '(*).txt' '$1.lis'

The parenthesis is the thing that gets replaced by the $1 (not the *’, as happens in mmv, and note the ``$’, not `=’, so that you need to quote both words).

zmv '(**/)(*).txt '$1$2.lis'

The same, but scanning through subdirectories. The $1 becomes the full path. Note that you need to write it like this; you can’t get away with ‘(**/*).txt’.

noglob zmv -W **/*.txt **/*.lis
zmv -w '**/*.txt' '$1$2.lis'

These are the lazy version of the one above; with -w, zsh inserts the parentheses for you in the search pattern, and with -W it also inserts the numbered variables for you in the replacement pattern. The catch in the first version is that you don’t need the / in the replacement pattern. (It’s not really a catch, since $1 can be empty.) Note that -W actually inserts ${1}, ${2}, etc., so it works even if you put a number after a wildcard (such as zmv -W ‘*1.txt’ ‘*2.txt’). [We alias zmv -[wW] to zm[wW]]

zmv -C '**/(*).txt' ~/save/'$1'.lis

Copy, instead of move, all .txt files in subdirectories to .lis files in the single directory `~/save’. Note that the ~ was not quoted. You can test things safely by using the `-n’ (no, not now) option. Clashes, where multiple files are renamed or copied to the same one, are picked up.

Here’s a more detailed description.

Use zsh pattern matching to move, copy or link files, depending on the last two characters of the function name. The general syntax is

zmv ‘<inpat>’ ‘<outstring>’

<inpat> is a globbing pattern, so it should be quoted to prevent it from immediate expansion, while <outstring> is a string that will be re-evaluated and hence may contain parameter substitutions, which should also be quoted. Each set of parentheses in <inpat> (apart from those around glob qualifiers, if you use the -Q option, and globbing flags) may be referred to by a positional parameter in <outstring>, i.e. the first (…) matched is given by $1, and so on. For example,

zmv '([a-z])(*).txt' '${(C)1}$2.txt'

renames algernon.txt to Algernon.txt, boris.txt to Boris.txt and so on. The original file matched can be referred to as $f in the second argument; accidental or deliberate use of other parameters is at owner’s risk and is not covered by the (non-existent) guarantee.

As usual in zsh, ’s don’t work inside parentheses. There is a special case for (**) and (***/): these have the expected effect that the entire relevant path will be substituted by the appropriate positional parameter.

There is a shortcut avoiding the use of parenthesis with the option -w (with wildcards), which picks out any expressions `*’, `?’, `<range>’ (<->, <1-10>, etc.), `[…]’, possibly followed by `#’s, `**/’, `***/’, and automatically parenthesises them. (You should quote any [’s or ]’s which appear inside […] and which do not come from ranges of the form `[:alpha:]’.) So for example, in

zmv -w '[[:upper:]]*' '${(L)1}$2'

the $1 refers to the expression `:upper:’ and the $2 refers to `*’. Thus this finds any file with an upper case first character and renames it to one with a lowercase first character. Note that any existing parentheses are active, too, so you must count accordingly. Furthermore, an expression like ‘(?)’ will be rewritten as ‘((?))’ — in other words, parenthesising of wildcards is independent of any existing parentheses.

Any file whose name is not changed by the substitution is simply ignored. Any error — a substitution resulted in an empty string, two substitutions gave the same result, the destination was an existing regular file and -f was not given — causes the entire function to abort without doing anything.

Bugs:

Parenthesised expressions can be confused with glob qualifiers, for example a trailing ‘(*)’ would be treated as a glob qualifier in ordinary globbing. This has proved so annoying that glob qualifiers are now turned off by default. To force the use of glob qualifiers, give the flag -Q.
1. The pattern is always treated as an extendedglob pattern. This can also be interpreted as a feature.

Unbugs:

You don’t need braces around the 1 in expressions like ‘$1t’ as non-positional parameters may not start with a number, although paranoiacs like the author will probably put them there anyway.

capitalize all vowels

zmv '(*)' '${1//(#m)[aeiou]/${(U)MATCH}}'

get rid of single quotes

zmv -nQ '*.html' "\${f//'/}"

with Q we need to escape certain things like $

lowercase and space-to-hyphen

zmv -n ‘*’ ‘${f:l:s/ /-}’

lowercase the directories

zmv -vw '**/(*)/' '${f:l}'

we had to run it 3 times though to get everything TODO

remove 1st four char

zmv '*' '$f[5,-1]' (where f is a zmv generated variable)

renumber files

i=0;zmv '(FILENAME-COMMON-PART)(*)' '$1$((i++)).mp4' #[] works too but may cause problems

i=1; zmv -n '(img*.jpg)' '${(l:2::0:)$((i++))}$1' #[] results in weird stuff
img[01,02,..].jpg

i=0;zmv -n '(bind.*)' '$1${(l:2::0:)}$((i++))' on bind.c, bind.h, bind.o
mv -- bind.c bind.c000
mv -- bind.h bind.h001
mv -- bind.o bind.o002

running other programs

zmv -p cp -o'-r' 'img/(*)' 'tmp/${1:u}'
zmv -C -o '-r' 'img/(*)' 'tmp/${1:u}'
either recursively copies contents of img/ to tmp/ uppercasing

zmW examples

Using zmv -w or zmW is very effective through globbing.

#  upcase the filename but not the extension
zmW '*.jpg' '${*:u}.jpg'

# pad left with zero
zmW 'img*.jpg' 'img${(l:2::0:)*}.jpg'

# replace spaces with hyphens
zmW '*' '${*// /-}'
better than
zmv '*-*' '${f// /-}'
zmv '*-*' '${f:gs/ /-}'

zoptions

-f  force overwriting of destination files.  Not currently passed
    down to the mv/cp/ln command due to vagaries of implementations
    (but you can use -o-f to do that).
-i  interactive: show each line to be executed and ask the user whether
    to execute it.  Y or y will execute it, anything else will skip it.
    Note that you just need to type one character.
-n  no execution: print what would happen, but don't do it.
-q  Turn bare glob qualifiers off:  now assumed by default, so this
    has no effect.
-Q  Force bare glob qualifiers on.  Don't turn this on unless you are
    actually using glob qualifiers in a pattern (see below).
-s  symbolic, passed down to ln; only works with zln or z?? -L.
-v  verbose: print line as it's being executed.
-o <optstring>
    <optstring> will be split into words and passed down verbatim
    to the cp, ln or mv called to perform the work.  It will probably
    begin with a `-'.
-p <program>
    Call <program> instead of cp, ln or mv.  Whatever it does, it should
    at least understand the form '<program> -- <oldname> <newname>',
    where <oldname> and <newname> are filenames generated. <program>
    will be split into words.
-P <program>
    As -p, but the program doesn't understand the "--" convention.
    In this case the file names must already be sane.
-w  Pick out wildcard parts of the pattern, as described above, and
    implicitly add parentheses for referring to them.
-W  Just like -w, with the addition of turning wildcards in the
    replacement pattern into sequential ${1} .. ${N} references.
-C
-L
-M  Force cp, ln or mv, respectively, regardless of the name of the
    function.

zz

specific computers

# broot setup for specific computers
if [[$HOST = 'schnell' || $HOST = 'adagio']];
   then
       #code
fi

bd (using tarrasch function for zsh because bd install doesn’t seem to work)
fzf a good addition too and works rather like the fuzzy find on broot
globbing seems to be inconsistent over the different versions of zsh
operations on multiple extensions ls *.{jpg,png}
list only directories (post) ls -d – *() ls -ld – *()

Computing on Towards Freedom

KDE

Plasma Global Shortcuts Reference

🗂️ Activities

🚀 Application Launchers

🪟 Window Management (KWin)

🧱 Tiling & Zones (Krohnkite & KZones)

🖥️ Desktop & Workspace Navigation

⌨️ Window Switching & Navigation

🔊 Audio, Media & Power

🖱️ Touchpad, Tablet & System

Musescore

implode

raggedy edge effect

hide empty bars

Vivaldi

reading list

cannot enter nickname

toggle extensions

tab stacking

exports

issue with systemd v249

lack of suitable h264 error

profile

sync

sort bookmarks

pdf viewing

dark theme

vivaldi social

Rosegarden

intro info

multiple voices

arpeggio symbol

installation components

rg without softsynth

rg with timidity

rg with yoshimi

yoshimi.rgd

post @folderol

TODO ornaments

rg lyrics entry AVOID

DONE rg -> ly

tour

handbook

1. Introduction

2. Rosegarden Files and Documents

3. The Track Editor

4. The Transport

5. The Studio

6. Special Parameters

7. Quantization

8. The Matrix editor

9. The Percussion Matrix editor

10. The Notation editor

11. The Event List editor

12. Customising Rosegarden

13. Credits and License

14. Rosegarden Revision History

zz

Lilypond

fundamental concepts

file structure

note structure

brackets and ties

voices

musescore transfer

automatic scaling to paper size

emacs

textedit

scheme

Netricks

mailto

No mailto

Simplest mailto

Throw in a subject

Email more than one recipient

A real opportunity!

Want to know more?

Imagemagick

Quick Fixes