Towards Freedom

Programmers are expected to program in a very dynamic and incremental manner, constantly re-evaluating existing Clojure definitions and adding new ones to their running applications.

We need to understand these better to make full use of CIDER.

• Interactive code evaluation can do this right in the code with C-c M-; often more useful than repl
• Powerful REPL
• Code completion
• Code reloading
• Definition & documentation lookup particularly excellent with C-c d d
• Enhanced syntax highlighting and indentation
• Enhanced error reporting
• clojure.test integration
• clojure.spec integration
• Interactive debugger
• Data Inspector
• Integration with Java logging frameworks
• Profiling & tracing
• ClojureScript support

Videos are good for learning CIDER.

Get familiar with interactive programming and cider-mode

Already done.

Most of this is already setup, but we should look at:

• Jacking-in without a Project just clj to get the repl or can also use in a dir M-x cider-jack-in-universal
• Connect to a Running nREPL Server after getting a repl running M-x cider-connect for a port

You can skip this section if you don’t plan to use cider-connect or don’t care about the advanced functionality that requires cider-nrepl. However, with tools.deps we can add to ~/.clojure/deps.edn

  :cider-clj {:extra-deps {cider/cider-nrepl {:mvn/version "0.49.0"}}
              :main-opts ["-m" "nrepl.cmdline" "--middleware" "[cider.nrepl/cider-middleware]"]}

  :cider-cljs {:extra-deps {org.clojure/clojurescript {:mvn/version "1.10.339"}
                            cider/cider-nrepl {:mvn/version "0.49.0"}
                            cider/piggieback {:mvn/version "0.5.3"}}
               :main-opts ["-m" "nrepl.cmdline" "--middleware"
                           "[cider.nrepl/cider-middleware,cider.piggieback/wrap-cljs-repl]"]}

And can then run M-x cider-connect to the repl.

• Overview Instead of repeatedly editing, compiling, and restarting an application, the programmer starts the application once and then adds and updates individual Clojure definitions as the program continues to run … some Clojure programmers have been known to keep a CIDER session running for weeks or even months!
• Implementation
• Demos
  Deep Dive into CIDER
  
  Emacs & Clojure, A Lispy Love Affair
  \\

C-h f cider-mode is useful, but even better is using which-key-mode to see all key possibilities.

• Basic: buffer (do first)
• Exotic: should be accessed via CIDER’s evaluation commands keymap, prefix C-c C-v
• Minibuffer: cider-read-and-eval C-c M-:
• Evaluation hooks: ??
• Synchronous vs Asynchronous: nrn
• Configuration: nrn
• Keybindings: C-M-x, C-c C-c, C-c C-v [C-]d

• Standard: TAB or M-TAB (setq tab-always-indent 'complete)
• Completion styles: user may want to use a different completion style just for the CIDER complete eg (add-to-list 'completion-category-overrides '(cider (styles basic)))
• Auto-completion: with company or corfu (latter already setup via codeium) - not sure if codeium conflicts with corfu or cider when doing its suggestions or incorporates those completions into the suggestions, but it is really not too important.

Clean everything up and reload one or more namespaces cider-ns-refresh, cider-ns-reload or cider-ns-reload-all

Probably NA.

Most of the time CIDER will display the errors in a dedicated buffer … At times, the error being displayed will originate from a bug in CIDER itself. Error displays can be configured.

CIDER presents stack traces in a special major mode, cider-stacktrace-mode. There is a lot that can be done here, so it is worth knowing how to use.

C-h . should be initial ref which provides the clojuredocs examples. However, C-c C-d C-d|w can also be useful because the window is easy to close and the latter goes right to clojuredocs. If you want the javadoc use C-c C-d C-j to see it in browser.

Search in docstrings of all loaded variables with C-c C-d C-e|f which can be quite useful if you know only some words in a doc, but don't really know what the item is.

Cross referencing other documentation can be done inside `` or in same namespace or outside (fqdn required). Useful if you’d like to be able to point other programmers at different definitions.

here is a vector of maps (def asym-parts [{:name "head" :size 3} {:name "mouth" :size 1} {:name "nose" :size 1} {:name "neck" :size 2} {:name "left-eye" :size 1} {:name "left-ear" :size 1}to {:name "left-shoulder" :size 3}])

match the :name to be "nose" and print that item TODO

an interesting solution to something similar https://stackoverflow.com/questions/18176372/clojure-get-map-key-by-value

(->> {:a "bar" :b "foo" :c "bar" :d "baz"} ; initial map (group-by val) ; sorted into a new map based on value of each key (#(get % "bar")) ; extract the entries that had value "bar" (map key)) ; get the keys that had value bar or (map key (#(get % "bar") (group-by val {:a "bar" :b "foo" :c "bar" :d "baz"})))

<2020-07-24 Fri> figured out how to install own work to .m2 started rewrite of algolyze using these ideas <2020-07-23 Thu> went through clojure style guide got better understanding of ns installed and worked with hylang - though we should check out cljpython too at some point

discussion on tech.ml was very promising with both gigasquid and joinr praising it. so that's what we'll go with for now - convert existing items to it and look for a visualization mechanism. <2020-07-22 Wed> got nuen.clj v1 done and testing.

<2020-07-18 Sat> go thru clojure reader is unsafe post about times post about project mechanism

<2020-07-17 Fri> nuen mostly done thx to help on clojurians

techml read into techml dataset a csv file nearly 3 million lines drop one column form new column the result of subtracting one column from another printout head of adjusted dataset

timings: lein run 62.93s user 0.86s system 123% cpu 51.800 total emacs repl "Elapsed time: 40112.441757 msecs" java -jar target/uberjar/techml-0.1.0-SNAPSHOT-standalone.jar 29.95s user 0.77s system 162% cpu 18.916 total

<2020-07-16 Thu> start nuen

rewrite anulysis for tech.ml rewrite anulysis for arrays

<2020-07-15 Wed> finish mikera and incanter build a matrix, dataset, array toolkit (mikera incanter and anything else) plan for clj cheatsheet explorations -> make own cheatsheet from it and others plan for java function access

will conversion to separate arrays be more efficient for subsequent processing? using arrays instead of incanter datasets may be practical?

<2020-07-14 Tue> finishing mikera and incanter we should go through cheatsheet items and understand how the main functions in the categories work

<2020-07-13 Mon> last couple of days were spent with julia programming. though the docs are excellent, emacs doesn't work too well with it right now. emacs does work well with R. so may be …

attempt to get python libraries on clojure using libpython-clj - but carin's example doesn't work completely. anyway, we don't really need to do this now because what we were interested in seeing was how well anulysis worked and we can actually do this better in other ways - such as plotting hlc dots with buffers from nu

checked out graalvm and that does look very interesting! again, we don't need it now, but it is definitely something to keep in mind for the future considering one demo showed 20 performance improvement. it may be useful even without its polyglot capabilities.

for now, here's what we focus on:

• learn mikera and incanter
• learn what we can do with direct access to java
• optimize anulysis
• get nuen completed

<2020-07-10 Fri> got R and candlesticks working - it's pretty good! main difficulty was that manjaro doesn't autoload make and gcc-fortran

plotly.clj doesn't seem to have candlesticks plan is to go through incanter plotting and compare to plotly.clj plotting

<2020-07-09 Thu> anulysis finish hopefully

plan for manual trading

1. examine situation around 12am
2. put in suitable entry limit or stop trades with sltp
3. try to exit trade around 12pm next day
4. close orders and start at 1 again

<2020-07-08 Wed> cider site debugging cider debugging tools debugging in clojure several items not quite understood but we now have a useable debugging mechanism

more learnt about loop-recur in that the main if statement can have various forms (eg let) in true or false. this knowledge enabled the best version to-date: (fn [rs nu-seq] (loop [nu nu-seq brs 0 prs 0 i 0] (if (or (> i (- dohlc-size part-size)) (empty? nu)) [brs prs] (let [nu0 (first nu-seq) hi ($ i :h dohlc) lo ($ i :l dohlc) cl ($ i :c dohlc) [is-brs is-prs] (if (= rs "r") [(brs? nu0 hi cl < -) (prs? nu0 hi cl > +)] [(brs? nu0 lo cl > +) (prs? nu0 lo cl < -)]) ] (cond is-brs (recur nu (inc brs) prs (inc i)) is-prs (recur (drop 1 nu) brs (inc prs) (inc i)) :else (recur nu brs prs (inc i)))))))

<2020-07-07 Tue> getting nu the partition mechanism provides far faster results than iterating over groups of 7 bars - even if you skip bars regularly because after a nu the latter three can never be a nu. however, the partitions don't provide anyway of keeping track of the index. a problem can develop in that as we go through nu we could be counting brs even before the occurence of the nu!

so here are 2 ideas: figure out the index of the nu through a loop-recur setup. something like loop ds to find index conj index+accumulatedindices to nu drop index+1 ds

similar idea, but we don't need to actually put the index anywhere. find index of nu drop to index+1 start to check for brs once prs drop current nu do over again this way we only start checking for brs after we find where the nu is

<2020-07-06 Mon> figured out how to use loop-recur better - it is a likeable combo and seems preferable to me than recursion now. got a good version of nu-finder done which includes index amongst other things and can be used outside anulysis. <2020-07-05 Sun> setup nu-orders.clj for populating nu with orders <2020-07-04 Sat> finish anulysis <2020-07-03 Fri> go through google hits example

<2020-07-02 Thu> determine whether to use core.matrix or incanter for most things incanter is being incorporated within core.matrix and it is not really developed much. so we'll make the switch and start looking at some of the other data science libraries.

discovered core.matrix creator mike anderson or mikera who has good youtube videos on the topic

seems as though core.matrix (most operations) and incanter (rw graphics etc) can work well together.

<2020-07-01 Wed> issues with datasets and matrices using map and apply figured out a lot about for loop <2020-06-30 Tue> dilemmas with figuring out how to work hilo with nus <2020-06-29 Mon> figuring out a more iterative anulysis <2020-06-28 Sun> work on sicp - understanding cond and if <2020-06-27 Sat> started sicp and INDUCTIVESYNTHESISOFFUNCTIONALPROGRAMS

<2020-06-26 Fri> figure out mechanism to compare hilo to stacked nu problem is that i keep wanting to put things into boxes rather than thinking functionally.

<2020-06-25 Thu> work on anulysis using stacks and proper influence.

(def d (ds/select-rows ds (range 9))) (ds/add-column d :sums (mt/add (sel d :cols :o) (sel d :cols :h))) (ds/add-column d :sums (mt/gt (sel d :cols :c) 83)) (ds/add-column d :sums (mt/mul (sel d :cols :c) (mt/gt (sel d :cols :c) 83))) ;crude

<2020-06-24 Wed> creating bounsr and penesr functions within let got involved with creating pairs

though there are some nice ideas in V1 of anulysis the overall attempt to have nu checked over the entire matrix is meaningless and produced absurdly high numbers - eg almost 8000 bounsr for nrow = 2248. so we need to restrict the nu influence to forward items only and have them flip after first penesr and disappear after a second penesr.

(defn anulysisV1 "analyzes pa near nu" [fp] (let [[ops his los cls fnm pf] (instrument-info fp) buffer (/ 12 pf) bounsr? (fn [nus hilos cls >< -+] (count (remove nil? (for [nu nus [hilo cl] (zip hilos cls)] (let [nu-buffer (-+ nu buffer)] (if (and (btn? hilo [nu nu-buffer]) (>< nu-buffer cl)) true)))))) bpsr? (fn [nus hilos ops cls >< -+] (for [nu nus [hilo op cl] (zip hilos ops cls)] (let [b-buffer (-+ nu buffer) p-buffer (-+ nu (- buffer))] (cond (and (btn? hilo [nu b-buffer]) (>< b-buffer cl)) :bounsr (and (btn? nu [op cl]) ) :penesr)))) part-size 5 midpt (quot part-size 2) hi-partition (partition part-size 1 his) lo-partition (partition part-size 1 los) nn (remove nil? (map #(nu-finder % max midpt) hi-partition)) uu (remove nil? (map #(nu-finder % min midpt) lo-partition)) boun-s (bounsr? uu los cls < +) boun-r (bounsr? nn his cls > -)

isbounsr #(= :bounsr %) ispenesr #(= :penesr %) bpu (bpsr? uu los ops cls < +) bpn (bpsr? nn his ops cls > -) ] [fnm boun-s boun-r (count (filter isbounsr bpu)) (count (filter isbounsr bpn)) (count (filter ispenesr bpu)) (count (filter ispenesr bpn))]))

(map anulysis (take 1 truefx-filepaths)) => (["audjpy1D.csv" 3829 3607]) => (["audjpy1D.csv" 4167 4035])

<2020-06-23 Tue> coined term lalwal lose a little win a lot rewrote outliers in truefx files - all were with the lows. some involving CHF may have been legitimate. anulysis almost done after figuring out that :while is not to be used with for - not sure what it is

<2020-06-22 Mon> started anulysis

<2020-06-21 Sun> get barchar working first idea of adjacent-confluence may work in reverse for shorter tf

started snippets in this file

made neat ns discovry - whatever is in core.clj including def/defn becomes available to every other file provided the namespace isn't changed! this really means that we can load up core.clj with whatever is required rather than import everything for every related program we create!

<2020-06-20 Sat> working with dataset functions add-column, reorder-column, remove-column wrote sel-col function which is useful to work with single column in dataset

discovered that sel on a dataset is very slow compared to sel on a matrix it seems to be far, far faster to convert a dataset to a matrix to do things and then convert it back if you want to utilize the variety of types a dataset offers. (see xpts in datmat.clj)

<2020-06-19 Fri> investigating various things all week without entry into diary many of clojure.core.matrix is replacing the incanter functions: see updates to incanter 2.0 https://github.com/incanter/incanter/wiki/Incanter-2.0-change-log so we investigate these in datmat.clj

<2020-06-11 Thu> get-orders-for done rearrange the ztut which was getting awkward as far as ns goes as well as other things. set it all up on gitlab

changing to ns works with C-c M-n after we C-c C-k the buffer. for some reason, (in-ns namespace) doesn't always work. this page is useful as may be other things on that site.

<2020-06-10 Wed> incanter things work nicely so far, but we need to become far more knowledgeable working with datasets, functions (what they do) and plotting.

also, the how i start thing is a bit confusing as carin jumps to things that we're not familiar with so we'll get back to that later.

<2020-06-09 Tue> incanter:

• work through data sorcery

  • cheatsheet use regularly to recall syntax and concepts
  • make use of api documentation throughout
  • namespace switching solved: repl takes main ns, but can switch with (in-ns 'whatever) so long as it is seen
• started How I Start which should be useful for namespace understanding and testing

<2020-06-08 Mon> incanter:

• work through ch6 Working with Incanter Datasets from Clojure Data Analysis Cookbook which was helpful though the datasets used were huge and therefore difficult to examine in some situations

<2020-06-07 Sun> installed inlein started thinking bayes

<2020-06-06 Sat> transferred from github to gitlab which seems to be smoother and cleaner. cleaned up tutorials into ztut folder cleaned up clj bookmarks went through excellent birdseye view of clj A Brief Beginner's Guide To Clojure

<2020-06-05 Fri> more cider it seems that form order is important because loading the buffer when dependencies don't come first makes eval impossible. cider has a lot of functions, but we need to figure out which ones we'll actually use successfully connected to an existing repl at localhost after getting port using: connection info C-c M-d

downloaded cheatsheets: clojuredocs

paredit:

cider:

<2020-06-04 Thu> got all of oanda ide to date as far as we had python going order fn greatly simplified - we never needed to use if-let

investigated cider further watched alchy bozbat

<2020-06-03 Wed> stuck on json not being parsed but that was because clojure follows the curl more closely using :body instead of :json whereas python didn't

<2020-06-02 Tue> get api functions and sring theory to where we were with python all api done as far as python progress except for the json issue

hobbit problem using reduce is neatest

excellent video with edmund jackson data science and clojure shows the landscape in 2012 and what options there were then probably worth following up on this.

<2020-06-01 Mon> understood the hobbit setup mostly in brave it is messy using loop, but much messier other ways

<2020-05-31 Sun> set up tut structures in clojure directory set up problems to solve file in to-solve.clj file

Clojure Libs and Namespaces: require, use, import and ns (require 'clojure.string 'clojure.test) (require '[clojure.string :as string]) to avoid (require ['clojure.string :as 'string]) (require 'clojure.string '[clojure.test :as test]) (require '(clojure string test)) (require '(clojure [string :as string] test))

consider the incanter lib issue (use '(incanter core stats charts io)) works right away and we can call view, histogram etc (require '(incanter core stats charts io)) doesn't seem to work at all, but that's due to ns issues: (incanter.core/view (incanter.charts/histogram (incanter.stats/sample-normal 1000))) is just fine

also see Clojure Namespaces and Vars for better ways as well as this answer from stackoverflow. we can solve this with (ns tut-incanter.core (:require (incanter [core :refer :all] [stats :refer :all] [charts :refer :all]))) though (use '(incanter core stats charts)) seems much simpler

both api requests with clj-http and json readability accomplished next step is to get functions to work

<2020-05-30 Sat> figuring out git mostly getting through brave ch3 except last exercise

<2020-05-29 Fri> started clojure for brave updated several youtube resources

<2020-05-28 Thu> finished cds introduction

<2020-05-27 Wed> halfway through cds introduction why clojure video by vijay is useful

<2020-05-26 Tue> continued with cds set up resources and videos watched russ olsen functional programming in 40 min

<2020-05-25 Mon> set up emacs for clojure watched rick hickey video on creating it got cider and lein working started cds tutorial