Default command dictionary

I have tested this on my end and I haven't found any bugs yet. I'll just submit this pull request so you can test yourself before merging. Let me know if you find any issues! Also, naturally, let me know any other suggestions or requests you may have.
Edit: add reference to enhancement implemented by this pull request: #8

I am going through it right now. Half done.
I would like to comment the default values for the parameters like  markdown_convert, formatted, code, and split
in  commands.yaml.example. E,g,  # Default value for code is false.
Maybe some lines can be removed as default is specified.

Excellent @wolterhv !
And in record time :) Today is weekend, you should be resting, ha ha ha ;)
Thanks a lot. This really makes things easier to understand for other people, and also easier to use!
Fantastic! Greatly appreciated.

As far as I can tell there are only 2 things left:

see above: adding comments to explain to user the 4 default values, and simplify yaml.example accordingly
remove: help.sh. It is no longer needed, right? All that info is already inside YAML file, correct? Any reason to keep it?

Anything else you can think of to clean up some more?

Regarding your question:

Since commands will be provided in a dictionary, should there be
a commands.yaml dictionary ready to use included? In this case,
I suppose we have to have a way to know what to populate the
paths node with. What is your opinion on a script to install files
to sensible default locations? Would you rather have the user
modify the commands.yaml and config.yaml files first,
before having a usable bot?

What do you think is best? What do you suggest?
Regarding the config.yaml.example file, I think a config.yaml.example is preferable and we force the user to copy, rename and modify the file.
I am inclined to think the same about the commands.yaml.example file. Make it obvious. Force the user copy/rename file and to set the paths. But I am open to suggestions. My std PATH includes all the necessary dirs, so in my personal case I would not have to set any additional paths in the yaml file. But that is just me.
But an action item results:

Should we decide to use the commands.yaml.example file, then we have to explain in the README file that this configuration step must be done manually, etc.

Interested to hear your opinion. Cheers!

Another idea comes to my mind: a "reload" feature. Entering the command reload into the bot, the bot re-reads the commands.yaml file and thereby adds or drops new commands without the need to restart the bot process. Live-update of commands so-to-speak. What do you think?

Thanks for your replies! See my replies below.

I would like to comment the default values for the parameters like markdown_convert, formatted, code, and split
in commands.yaml.example. E,g, # Default value for code is false.
Maybe some lines can be removed as default is specified.

Ok I'll look into that.

And in record time :) Today is weekend, you should be resting, ha ha ha ;)
Thanks a lot. This really makes things easier to understand for other people, and also easier to use!
Fantastic! Greatly appreciated.

I gladly use my weekends for hobby work :) And this is just a small addition to your great project!

see comment above: adding comments to explain to user the 4 default values, and simplify yaml.example accordingly
remove: help.sh. It is no longer needed, right? All that info is already inside YAML file, correct? Any reason to keep it?
see comment below: possibly adjust README.md file to explain to user how to "install"/"modify" the commands.yaml.example file.

Anything else you can think of to clean up some more? Any other little things that should be done, are missing?

I tried explaining what they do on the commands.yaml.example , lines 19-29 in revision adf098a. Is this what you meant? I'm not sure what formatted does so I left that one blank and forgot to update it later. Does it enable formatting of a subset of HTML?
Regarding the help.sh script, you're totally right, I forgot about that. I'll push the commit where I delete the file soon.
Regarding the commands.yaml.example file, I agree with you. It is simplest for the project to have the user just read the instructions and install and edit the YAML files as best fits them. The action here is then to update the README.md.
Regarding further cleanups and enhancements, I think I'm happy to merge this for now and maybe open some enhancement requests as the ideas come :) Off the top of my head, maybe I would propose unifying some commands which do similar things, like the tides and waves commands or those commands to retrieve cryptocurrency prices.

Another idea comes to my mind: a "reload" feature. [...]

I think this is a great idea, it can help the user debug their scripts a lot quicker than they would should they need to restart the bot every time. I like it. I think we should open a separate issue for that though.
I hope I addressed all your points. Let me know if I missed anything!

I would like to comment the default values for the parameters like markdown_convert, formatted, code, and split
in commands.yaml.example. E,g, # Default value for code is false.

Maybe some lines can be removed as default is specified.

Ok I'll look into that.

I see that since there is only one call to self._os_cmd in bot_commands.py which is used to execute a command from the command dictionary, I can't think of a way to use the parameter defaults in some calls and the values from the command dictionary in others. Therefore, I'm not sure whether I can remove the DEFAULT_OPT_ constants from the definition of the CommandDict class. I also see that default values are defined in self._os_cmd and in send_text_to_room. Where would you suggest centrally located defaults to live?

see comment above: adding comments to explain to user the 4 default values, and simplify yaml.example accordingly
remove: help.sh. It is no longer needed, right? All that info is already inside YAML file, correct? Any reason to keep it?
see comment below: possibly adjust README.md file to explain to user how to "install"/"modify" the commands.yaml.example file. missing?

I tried explaining what they do on the commands.yaml.example , lines 19-29 in revision adf098a.
Is this what you meant? I'm not sure what formatted does so I left that one blank and forgot to
update it later. Does it enable formatting of a subset of HTML?

I did not explain myself well. It is taken care of., It was a matter of documentation, explanation.
Please see my changes in commands.yaml.example. Please review.
So, if you agree this bullet point it resolved, taken care off.

Regarding the help.sh script, you're totally right, I forgot about that. I'll push the commit where I delete the file soon.

Taken care off, I just removed it. Done.

Regarding the commands.yaml.example file, I agree with you. It is simplest for the project to have the
user just read the instructions and install and edit the YAML files as best fits them. The action here is
then to update the README.md.

Excellent. We agree. So, we fix the RERADME.md file (TODO #1)

Regarding further cleanups and enhancements, I think I'm happy to merge this
for now and maybe open some enhancement requests as the ideas come :)
Off the top of my head, maybe I would propose unifying some commands
which do similar things, like the tides and waves commands or those
commands to retrieve cryptocurrency prices.

I personally see value in having these items separated and rather small,
like small Lego blocks. Why? Because this was it is more customizable to the bot users.
Imagine, your daughter just wants to surf and only cares about waves but your grandma
only cares about walking on the beach and only cares about tides.
Or your friend A only has crypto X, your friend B only crypto Y and friend C only crypto Z.
So, I would rather NOT combine the commands so each user can get a personalized "fit".

Another idea comes to my mind: a "reload" feature. [...]

I think this is a great idea, it can help the user debug their scripts a lot
quicker than they would should they need to restart the bot every time. I like it.
I think we should open a separate issue for that though.

I opened issue #10
So this is "done" as in "it is placed on the TODO list in issues page".

I hope I addressed all your points. Let me know if I missed anything!
You did.

I changed some files (command_dict.py, commands.yaml.example, etc).
Please "pull" before making new changes so that we do not have to rebase or step on each others toes.

A question: Why did you use lowercase "true" in Yaml file instead of uppercase "True" ? Same question for false/False.

I would like to comment the default values for the parameters like markdown_convert, formatted, code, and split
in commands.yaml.example. E,g, # Default value for code is false.

Maybe some lines can be removed as default is specified.

I see that since there is only one call to self._os_cmd in bot_commands.py which is used to execute a command from the command dictionary, I can't think of a way to use the parameter defaults in some calls and the values from the command dictionary in others. Therefore, I'm not sure whether I can remove the DEFAULT_OPT_ constants from the definition of the CommandDict class. I also see that default values are defined in self._os_cmd and in send_text_to_room. Where would you suggest centrally located defaults to live?

OK. Again I did not explain myself well. My issue is a lot simpler. All I am saying is:
In file commands.yaml.example we have this
  # see also: upgrade
  check:
    regex:             "^check$|^chk$|^status$|^state$|^check .*$|^chk .*|^status .*$|^state .*$|^check.sh$|^check.sh .*",
    cmd:               check.sh
    help:              check status, health status, updates, etc.
    markdown_convert:  false
    formatted:         true
    code:              false

Here we can remove the last 2 lines. We can remove line formatted: true and line code: false because we are setting these 2 values to the default.
You agree that in our example these 2 lines can be removed? The code will do the same thing, right?

Question: Shall we remove them?
As advantage: Less code, simpler, etc.
But as disadvantage: less readable, people might not know the defaults and might find it easier to just have the values there anyway.

So, TODO # 2 : decide and possibly remove some lines.

Wow, that was way too lengthy. Sorry, for being so wordy.
Summary:

TODO # 1 : update README.md, give install instructions, ...
TODO # 2: decide if we want to remove some lines from yaml file where we assign the default values
Issue #10 : the "reload" feature for later ...

Thanks again. Did I miss anything? I can do the README ( TODO # 1)


Feature: DMAMUX support

G0 is different from other STM32 MCUs because this uses a thing called DMAMUX. Well actually H7, L4+ and WB series are also using it but currently it is not implemented in their Rust HALs. So we can be the first ones to tackle this problem.
If this is done it allows to port a lot of DMA based drivers from stm32l0xx-hal which are relatively mature.
27.05.2020: Recently rust embedded wg suggested that this is way to go with DMA: https://github.com/ra-kete/dma-poc.
And an example how it is used: https://github.com/stm32-rs/stm32f3xx-hal/pull/86

Do you have some design in mind already? I have a project now where I need uart+dma, so maybe I can help a bit.

I do not have any particular design in mind yet and I would be very happy with PRs about this feature.

Do you by any chance have Rust code using DMAMUX without using HAL? Just the PAC. Otherwise I think that's something I will start with.
I found this application note AN5224.

Yes, I have written specialized ADC driver with hardcoded channels that is using DMAMUX.
use cortex_m::asm;
use hal::stm32;
use hal::prelude::*;
use hal::analog::adc::{Precision, SampleTime};
use hal::timer::Timer;
use hal::time::Hertz;

pub fn init(adc_dma_buf: &[u16; 6]) {
    // notice that G0 HAL does not have ADC DMA support yet, therefore it is done here manually

    let rcc = unsafe { &(*stm32::RCC::ptr()) };
    let adc = unsafe { &(*stm32::ADC::ptr()) };
    let dma = unsafe { &(*stm32::DMA::ptr()) };
    let dmamux = unsafe { &(*stm32::DMAMUX::ptr()) };

    // set ADC clock source to System clock
    rcc.ccipr.modify(|_, w| unsafe { w.adcsel().bits(0b00) });
    // enable ADC clock
    rcc.apbenr2.modify(|_, w| w.adcen().set_bit());

    // async ADC clock can be sourced from SYSCLK, HSI16 and PLLP
    // it can be prescaled using ADC_CCR PRESC[3:0] from 1 to 256
    // however async clock introduces jitter if ADC conversions are triggered using
    // external trigger like timer
    // therefore here ADC clock is derived from APB PCLK/4
    // notice that for this synchronous source ADC_CCR PRESC[3:0] cannot be used as prescaler
    // if SYSCLOCK is 64 MHz and APB Prescaler is 1 then PCLK is also 64 MHz
    // and ADC clock is PCLK/4 = 64/4 = 16 MHz
    adc.cfgr2.write(|w| unsafe { w.ckmode().bits(0b10) });

    // enable ADC voltage regulator
    adc.cr.modify(|_, w| w.advregen().set_bit());
    for _ in 0 .. 1_000 { asm::nop() }

    // perform calibration
    adc.cr.modify(|_, w| w.adcal().set_bit());
    while adc.isr.read().eocal().bit_is_clear() {}

    // configure ADC settings
    unsafe {
        adc.cfgr1.write(|w|
            w
            .res().bits(Precision::B_12 as u8)
            // discontinuous mode, software or hardware event is needed to start sequence conversion
            .discen().clear_bit()
            // right aligned
            .align().clear_bit()
            // DMA circular mode
            .dmacfg().set_bit()
            // generate DMA requests
            .dmaen().set_bit()
            // select TIM6_TRGO as hardware trigger
            .extsel().bits(0b101)
            // enable hardware trigger on rising edge
            .exten().bits(0b01)
        );

        // set sampling time to 40
        // tconv = Sampling time + 12.5 x ADC clock cycles
        // with 16 MHz clock it is:
        // tconv = (40 + 12.5) * ADC clock cycles = 92.5 * 0.063 = 3.281 us per channel
        adc.smpr.modify(|_, w| w.smp1().bits(SampleTime::T_40 as u8));
    }

    // select ADC channels
    adc.chselr().write(|w| unsafe { w.chsel().bits(0x4B9) });

    // reset DMA
    rcc.ahbrstr.modify(|_, w| w.dmarst().set_bit());
    rcc.ahbrstr.modify(|_, w| w.dmarst().clear_bit());
    // enable DMA clock
    rcc.ahbenr.modify(|_,w| w.dmaen().set_bit());

    // setup DMA channel 1 to read out ADC data
    unsafe {
        dma.ccr1.write( |w|
                w
                // priority level 2
                .pl().bits(0b10)
                // memory/peripheral request size: 16-bit (1)
                .msize().bits(0b01)
                .psize().bits(0b01)
                // cirular mode
                .circ().set_bit()
                // increment memory ptr, do not increment periph ptr
                .minc().set_bit()
                .pinc().clear_bit()
                // disable 'memory-to-memory' mode
                .mem2mem().clear_bit()
                // use 'peripheral -> memory' transfer direction
                .dir().clear_bit()
                // enable only transfer complete interrupt
                .teie().clear_bit()
                .htie().clear_bit()
                .tcie().set_bit()
            );

        // route DMA channel 1 to ADC - on STM32G0 it is done using DMAMUX
        // notice that DMAMUX is 0-indexed, however DMA channel is 1 indexed
        // check dmareq_id from reference manual: "table 42. DMAMUX: assignment of multiplexer inputs to resources"
        // so 5 means use ADC as DMA input source
        dmamux.dmamux_c0cr.write( |w| w.dmareq_id().bits(5).ege().set_bit());

        // setup DMA to transfer data from ADC_DR register to adc_dma_buf
        let adc_data_register_addr = &adc.dr as *const _ as u32;
        let adc_dma_buf_addr : u32 = adc_dma_buf as *const [u16; 6] as u32;
        dma.cndtr1.write( |w| w.ndt().bits(adc_dma_buf.len() as u16));
        dma.cpar1.write( |w| w.pa().bits(adc_data_register_addr) );
        dma.cmar1.write( |w| w.ma().bits(adc_dma_buf_addr));
    }
}

pub fn start(dma: &mut stm32::DMA, adc: &mut stm32::ADC, mut trigger: Timer<hal::stm32::TIM6>, samplerate: Hertz) {
    // enable DMA
    dma.ccr1.modify(|_r,w| w.en().set_bit());

    // enable ADC
    adc.isr.modify(|_, w| w.adrdy().set_bit());
    adc.cr.modify(|_, w| w.aden().set_bit());
    while adc.isr.read().adrdy().bit_is_clear() {}

    // ADC conversions are started using TIM6 TRGO event
    let tim6 = unsafe { &(*stm32::TIM6::ptr()) };
    // update mode - the update event is selected as a trigger output (TRGO)
    tim6.cr2.modify(|_, w| unsafe { w.mms().bits(0b010) });
    trigger.start(samplerate);

    // start listening TRGO events
    adc.cr.modify(|_, w| w.adstart().set_bit());
}

pub fn raw_to_mv(raw: u16) -> u16 {
    let vref: u32 = 3300;
    (raw as u32 * vref / 4096) as u16
}

I managed to make a working UART - DMA transfer with help of your example, see this gist.
So I kinda understand now what DMAMUX is for. However the manual mentions you can also do peripheral-to-peripheral DMA transfers. But it is not clear to me how you can configure that. Since the DMAMUX CxCR register only requires 1 DMAREQ_ID table 36.

I think it is ok if peripheral-to-peripheral transfers are not supported in first version. If we could to peripheral-to-memory and memory-to-peripheral then pretty much everything is covered (UART, ADC, SPI, I2C, etc).

Dmamux support is now merged.


Fix Initialize original_tablet_schema in _expand_variant_t…

…o_subcolumns to address potential nullptr issue (#32184)
Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run build all

run buildall


Updating fix_link to get rid of space

I also added an inspect-link flag so that you can easily get all the http links printed out which could help with making sure that they are up to date.
Closes #64

Yeah it was prettify. I found an SO answer about it. I was surprised it wasn't easier to find info about.

Good sleuthing!


Hugo server stops working after repeated restarts

What version of Hugo are you using (hugo version)?

$ hugo version
Hugo Static Site Generator v0.75.1-A4A7BAB7 linux/amd64 BuildDate: 2020-09-15T06:46:04Z

(docker hugo:ubuntu)
So I basically just went through the quickstart tutorial and now I tried out a bunch of themes. So I installed multiple themes as git submodules. For this reason I ended up repeatedly stopping and starting the server (inside a docker container).
The site started to load slower and slower until it just did not load at all. The F12 Network says that a lot of files were never received.
I tried

passing --network=host to the docker container. (Maybe the network is the problem - now it shouldn't be (I think))
repeatedly reloading with F5
deactivate the browser cache in F12
repeatedly reload again

it just did not reload. At that point it was late in the evening, so I just called it a day. Curiously I tried again today and things worked again in the beginning until it started to slow down again. Now firefox simply refuses to load the site.
During the writing of this report I realized that I have not yet tried a different browser: apparently edge works right now. I will give an update if edge shows the same behavior after some time.
It might also be a chache issue with firefox? I don't know.

@FelixBenning Can we close this issue?

forgot about this - I think I just installed hugo outside docker at some point did not encounter any issues there. No idea if docker is still acting up, maybe it is related to resource constraints within a docker container. Close it if you want :)


Issues triage question -- still under development?

Hey I really like this library! And it looks like others do as well given the amount of open issues. I'm curious about the development status on the library and capacity. I even saw another issue with a company interested in explicitly funding active development.
In essence, how are maintainers feeling about issue triage? is this repo in a place to start a new project with?

never mind, we're going a different direction

What is the different direction? @JackHowa


Project organisation for variability of targets

How to solve the problem of having Dockerfiles for multiple versions of a library?

What about different architectures?
What about different variants? E.g. Tesseract with/without support for pango, cairo etc.
How to go about organizing without bleeding out the entire param interface of ./configure?

Here's a pickle:
By default e.g. tesseract depends on leptonica, which depends on tiff. That means by default, tesseract will now also depend on tiff being present at runtime, because it will be loaded through leptonica.
In this case, it would be nice, if there was a way to specify a version of leptonica, compiled without tiff as a dependency, as a dependency for tesseract. E.g. a variant of a different name, leptonica-no-tiff.


Updated widgets.py

Changed how PagedownWidget.render() method works.
Added conditional assigining to context variable based on django version.
See https://github.com/timmyomahony/django-pagedown/issues/42

Hey, apologies for the delay. I'll check this out

@sobolevn You beat me to it! Thanks for taking care of this. 👍


[NFC] Fix typo in SerializedModuleLoader.cpp

No functional change, just a typo fixed in a function doc comment.

@swift-ci please smoke test and merge