SLO Report Output update

This Pull Request updates the SLO report output formats.
Tabbed:

JSON:
$ reliably slo report --format json
{
  "api_version": "1.0rc",
  "timestamp": "2021-04-01T15:50:31.291065Z",
  "dependencies": [],
  "service_level": {
    "target": {
      "error_percent": 1,
      "latency_ms": 300
    },
    "actual": {
      "error_percent": 0,
      "latency_ms": 761
    },
    "delta": {
      "error_percent": -1,
      "latency_ms": 461
    }
  },
  "window": {
    "from": "2021-03-31T16:50:31.298781+01:00",
    "to": "2021-04-01T16:50:31.298781+01:00"
  }
}

Simple:
$ reliably slo report --format simple
WARN[0001] You are under your error budget by 1.00 percent. You could tighten your budget, or could decrease the quality of the experience your application provides (e.g by reducing the amount of resources given to your application).
WARN[0001] The average latency threshold has been exceeeded by 461ms

Help:
$ reliably slo report --help
Report my slo metrics

Usage:
  reliably slo report [flags]

Flags:
      --format string          specify report output format. Allowed Values: [json, simple, tabbed] (default "tabbed")
  -h, --help                   help for report
  -f, --manifest-file string   the path to the manifest file (default "reliably.yaml")
  -o, --output string          where the report should be written to

Global Flags:
      --no-color   Disable color output
  -v, --verbose    verbose output

Environment variables:
  See 'reliably environment --help' for the list of supported environment variables.

Feedback:
  You can provide with feedback or report an issue at https://github.com/reliablyhq/cli/issues/new

Resolves: #139

dependency update
added simple, tabbed and json report formatting logic, initial commit
added observationwindow to track report window duration
removed unecessary functions, adjusted formatting

@daidokoro we made a few improvements with @mperrien to the simple format and fixed the missing latency SLO in the report , when it was Ok.
Overall looks great !!
I'll merge it now, thanks


feat: Implement HbarLimitService#addExpense

Description
This PR implements the addExpense method in the HbarLimitService class and adds corresponding unit tests. The changes include:

Implementing the addExpense method to handle expense addition for ETH addresses.
Adding unit tests for the addExpense method to ensure correct functionality.
Adding unit tests for other methods in HbarLimitService to improve test coverage.

Changes

Implementation of addExpense Method:

Added logic to create a basic spending plan if none exists for the provided ETH or IP address.
Updated the remaining budget and spending history after adding an expense.
Added error handling for cases where neither ETH nor IP address is provided.

Unit Tests:

Added tests for the addExpense method to cover various scenarios, including:

Adding an expense when both ETH and IP addresses are provided.
Adding an expense when only ETH address is provided.
Handling errors when adding an expense fails.

Added tests for other methods in HbarLimitService to ensure comprehensive test coverage.

Related Issues
Fixes #2901
Notes for Reviewer

Ensure to implement the remaining TODOs related to IP address handling in future PRs.
Review the added unit tests to confirm they cover all edge cases and scenarios.

A couple of nits, but I have a more general question.
I assume based on the description of issue #2896 that the plan will be to simply swap out the old HBar Rate Limiter with the new HBar Rate Limiter in the sdkClient.ts. If that is the case what happens if the Tier 3 general users are limited to 10K HBar per day, and a given partner is limited to 20K per day, and the general users use up all of the 10K budget before the partner has used up all of their allocated 20K. Is the partner rate limited as well, even though the partner may have only used up 12K Hbar that day?
Should we have a test for this?

I have also been thinking about this, but I couldn't come up with something else than eliminating the total daily limit and only keeping the tiered limits per user/partner, let's discuss it on the parking lot today.


tiller pod unhealthy always restart

describe tiller pod show
Events:
  Type     Reason     Age                    From                Message
  ----     ------     ----                   ----                -------
  Warning  Unhealthy  33m (x2 over 38m)      kubelet, k8s-node1  Readiness probe failed: Get http://<IP_ADDRESS>:44135/readiness: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
  Warning  Unhealthy  32m (x3 over 4h5m)     kubelet, k8s-node1  Liveness probe failed: Get http://<IP_ADDRESS>:44135/liveness: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
  Normal   Pulled     26m (x2 over 5h4m)     kubelet, k8s-node1  Container image "gcr.io/kubernetes-helm/tiller:v2.13.1" already present on machine
  Normal   Killing    26m (x2 over 32m)      kubelet, k8s-node1  Container tiller failed liveness probe, will be restarted
  Normal   Created    26m (x2 over 5h4m)     kubelet, k8s-node1  Created container tiller
  Warning  Unhealthy  26m (x3 over 5h4m)     kubelet, k8s-node1  Liveness probe failed: Get http://<IP_ADDRESS>:44135/liveness: dial tcp <IP_ADDRESS>:44135: connect: connection refused
  Warning  Unhealthy  26m (x2 over 32m)      kubelet, k8s-node1  Liveness probe failed: Get http://<IP_ADDRESS>:44135/liveness: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
  Normal   Started    18m (x5 over 5h4m)     kubelet, k8s-node1  Started container tiller
  Warning  Unhealthy  13m (x14 over 38m)     kubelet, k8s-node1  Readiness probe failed: Get http://<IP_ADDRESS>:44135/readiness: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
  Warning  Failed     8m2s                   kubelet, k8s-node1  Error: context deadline exceeded
  Warning  Unhealthy  3m48s (x41 over 5h4m)  kubelet, k8s-node1  Readiness probe failed: Get http://<IP_ADDRESS>:44135/readiness: dial tcp <IP_ADDRESS>:44135: connect: connection refused

Output of helm version:
k8s1@k8s-master:~$ helm version
Client: &version.Version{SemVer:"v2.13.1", GitCommit:"618447cbf203d147601b4b9bd7f8c37a5d39fbb4", GitTreeState:"clean"}
Error: could not find a ready tiller pod

sometime tiller is right
k8s1@k8s-master:~$ helm version
Client: &version.Version{SemVer:"v2.13.1", GitCommit:"618447cbf203d147601b4b9bd7f8c37a5d39fbb4", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.13.1", GitCommit:"618447cbf203d147601b4b9bd7f8c37a5d39fbb4", GitTreeState:"clean"}

Output of kubectl version:
k8s1@k8s-master:~$ kubectl version
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.0", GitCommit:"641856db18352033a0d96dbc99153fa3b27298e5", GitTreeState:"clean", BuildDate:"2019-03-25T15:53:57Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.0", GitCommit:"641856db18352033a0d96dbc99153fa3b27298e5", GitTreeState:"clean", BuildDate:"2019-03-25T15:45:25Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/amd64"}

Cloud Provider/Platform (AKS, GKE, Minikube etc.):

I'm going to close this ticket due to inactivity, but please re-open if this still needs to be addressed. Thanks!


use multiple values for the fees accrued in test_checkpoint

As of now it is hard-coded to zero.

Closing since parity efforts are moving to the Rust implementation


Customize URL

It is possible to customize the URL, to give a complete form to the root, for example instead of
<IP_ADDRESS>:8000/app_two/model_1/list
to be
<IP_ADDRESS>:8000/api/list
Thank you so much

I think so but Instead of defining the urls for the whole app like this:
urlpatterns += crud_for_app('testapp')

You should define the urls for every CRUD resource like this:
    url(r'/api/author', include(authorcrud.get_urls())),
    url(r'/api/invoice', include(invoicecrud.get_urls())),
    url(r'/api/customer', include(customercrud.get_urls())),

And you will get urls like:

api/author/list
api/author/create
...

Please take a look to the documentation and the testapp sample in the repo:

https://django-cruds-adminlte.readthedocs.io/en/latest/installation.html#urls-for-the-crud
https://github.com/oscarmlage/django-cruds-adminlte/blob/master/demo/demo/urls.py#L36

Dunno if it fits for you, feedback is always welcome ;)

Maybe a good idea is be able to pass the url you want to get_urls() method to customize not only at model level but at resource level too. i.e.:
url(r'/my/custom/list', include(authorcrud.get_urls('list'))),
url(r'/my/custom/del', include(authorcrud.get_urls('delete'))),

What do you think guys? @luisza @migue56 @mbourqui

this solution does not work, gives the following error and the server does not sell
path('api/ina/', include(MotivoInasistenciaCRUDView().get_urls('list'))),
path('api/ina/', include(MotivoInasistenciaCRUDView().get_urls('list'))),

TypeError: get_urls() takes 1 positional argument but 2 were given
in this other
path('api/ina/', include(MotivoInasistenciaCRUDView().get_urls())),
I form this url
api/ina/

this solution does not work, gives the following error and the server does not sell

No, I don't mean it's working now like that, I mean that could be a good idea to extend the functionality -as a new feature- if you call get_urls() with an extra argument.

Ok, sorry yes that great
Thanks

From a conceptual point of view, I think it makes sense, as you may not want to expose the full CRUD urls. Or as mentioned, customize only one or the other resource.
But I haven't check how feasible it would be.

I think that it could be nice to customize the URL and the URL name we can achieve this by making a dict attribute.
If dict is None then the "normal" behavior, else if the dict have the key "list" : { "name" = "auth_group_list", "url" : "/auth/group/list" }
It would be nice and easy I think :)

I'm just getting this page it's something wrong in URL can u plz solve this


Create canonically-named logger for pyjavaz.bridge to use

As described in https://github.com/micro-manager/pycro-manager/issues/766#issuecomment-2147885614, this is my recommended fix to logging here, which still allows users to control pyjavaz logging output as-needed.
Note that this is technically a bug fix, since it removes usage of pycromanager, which is not declared in our requirements here---(also this avoids a tricky circular import issue).

Awesome, thanks for the fix. I'll bump the version and push to pypi


Fix linking of stablehlo_capi_objects

Looks like I hit the same issue: https://github.com/openxla/stablehlo/pull/2528
I think my fix is the correct one.

reviewed https://github.com/openxla/stablehlo/pull/2528. Feel free to close this one as duplicate?


Add wiznet_w5100s_evb_pico support

#665 Change the Branch for PR

The schematic at https://www.mouser.co.uk/new/wiznet/wiznet-w5100sevbpico-eval-board/ suggests that there's an LED connected to GPIO25, so you could probably add:
#ifndef PICO_DEFAULT_LED_PIN
#define PICO_DEFAULT_LED_PIN 25
#endif

?
Also, you might want to add some defines for your INTn and RSTn pins? e.g.
#define WIZNET_W5100S_EVB_PICO_RSTN_PIN 20
#define WIZNET_W5100S_EVB_PICO_INTN_PIN 21

:man_shrugging:  (see https://github.com/raspberrypi/pico-sdk/blob/master/src/boards/include/boards/pimoroni_keybow2040.h for something similar)

@kilograham I added defined pins that are board dependent.

I added an blank line for readability.

Don't you have the action I need to do?

is this good @lurch

I approved it 22 days ago :wink:

i was looking at the previous comment from 12 days ago, but I will take it as you are happy


Color of links in block quotes same as color of normal text.

See above. Can't see links in block quotes unless you hover your mouse over them. Thanks! :)

Yes, this issue is known and quite bothersome. Hoping to figure out a solution soon.

If anyone out there can fix it, please go for it!

Is this still a problem? If so, please attach a screenshot so I know where to look 😄
On my end it looks like this:


Adding rule property to pcs location

Overview
On our infrastructure we need to add some rule to manage location constraints with pcs. But this feature wasn't supported by the puppetlabs-corosync module.
Here is my Pull Request to add this functionnality.
How to do it manually
To add a rule, you need to execute the following commands :
/usr/sbin/pcs constraint location add location-sv-dummy sv-dummy1 node_name.domain INFINITY
/usr/sbin/pcs constraint location sv-dummy1 rule score=-INFINITY #kind eq container
That gives us this xml output from pcs cluster cib command:
      <rsc_location id="location-sv-dummy" node="node_name.domain" rsc="sv-dummy1" score="INFINITY"/>
      <rsc_location id="location-sv-dummy1" rsc="sv-dummy1">
        <rule id="location-sv-dummy1-rule" score="-INFINITY">
          <expression attribute="#kind" id="location-sv-dummy1-rule-expr" operation="eq" value="container"/>
        </rule>
      </rsc_location>

And you should obtain the following output from pcs constraint location show command:
  Resource: sv-dummy1
    Enabled on: node_name.domain (score:INFINITY)
    Constraint: location-sv-dummy1
      Rule: score=-INFINITY
        Expression: #kind eq container

This rule means that you deny (-INFINITY) to launch your resource on a container (like VM).
How to manage it with Puppet
Example of how you can set your rule with the following Puppet code.
    cs_location { 'location-sv-dummy':
      primitive => 'sv-dummy1',
      node_name => 'node_name.domain',
      score     => 'INFINITY',
      rule      => { 'score'      => '-INFINITY',
                     'expression' => { 'attribute' => '#kind',
                                       'operation' => 'eq',
                                       'value'     => 'container'
                     }
                   }
    }

By default,  the rule => score is set to score=INFINITY (see pcs man page). You can also use score-attributes instead of score.
The expression determines the operation to execute for this rule.
When you applied it, it works fine:
    Debug: Cs_location[location-sv-dummy](provider=pcs): {}
    Notice: /Stage[main]/Main/Cs_location[location-sv-dummy]/ensure: created
    Debug: Executing '/usr/sbin/pcs constraint location add location-sv-dummy sv-dummy1 node_name.domain INFINITY'
    Debug: Executing '/usr/sbin/pcs constraint location sv-dummy1 rule score=-INFINITY #kind eq container'
    Debug: /Stage[main]/Main/Cs_location[location-sv-dummy]: The container Class[Main] will propagate my refresh event
    Debug: Class[Main]: The container Stage[main] will propagate my refresh event

Issues
There is still a problem with my code and I didn't find how to solve it yet. This problem appears when you apply for the second time your configuration.
Here is the Puppet output for this issue :
    Debug: Cs_location[location-sv-dummy](provider=pcs): {:name=>"location-sv-dummy", :ensure=>:present, :primitive=>"sv-dummy1", :node_name=>"node_name.domain", :score=>"INFINITY", :provider=>:pcs, :rule=>{}}
    Notice: /Stage[main]/Main/Cs_location[location-sv-dummy]/rule: rule changed {} to '{"score"=>"-INFINITY", "expression"=>{"attribute"=>"#kind", "operation"=>"eq", "value"=>"container"}}'
    Debug: Executing '/usr/sbin/pcs constraint location add location-sv-dummy sv-dummy1 node_name.domain INFINITY'
    Debug: Executing '/usr/sbin/pcs constraint location sv-dummy1 rule score=-INFINITY #kind eq container'
    Error: /Stage[main]/Main/Cs_location[location-sv-dummy]: Could not evaluate: Execution of '/usr/sbin/pcs constraint location sv-dummy1 rule score=-INFINITY #kind eq container' returned 1:

I suppose that the data obtains from the xml output are not correctly set on my rule variable but I didn't know how to solve it, so don't hesitate to send me your ideas if you can solve it ^^
Thanks for your help.
Julien Georges<EMAIL_ADDRESS>

Hi Julien, I had similiar needs, but my implementation is different, what do you think?
https://github.com/gavmain/puppetlabs-corosync#configuring-locations

@gavmain is this #136 ?
Before review on either PR commences, it would be good to have feedback from you and @GiooDev  whether either of them should be preferred.
/cc @igalic

Hey @ffrank, yes it is. I was working under the guidance of @hunner. There a few differences with my code, based on hacking around with the CIB:

You can only declare a node-based or rule-based resource, I have coded that logic in.
Score should only be declared once. the score parameter is located in different places in the CIB XML depending on if the resource is rule-based or node-based.
boolean-op needs to be accounted for, as the rule can use OR/AND.
Based on my chat with @hunner, he pushed me to use an array for the rule to maintain order.

I believe that declaring one cs_primitive, should only create one rsc_location entry. @GiooDev 's implementation creates two.
I am open to debate :)

I note both only have a pcs provider, not crm. Any plans to add crm support?

I could be persuaded to. I don't have Debian running at home, so it would take me an afternoon to get a couple of nodes up and have the code developed. I was hoping someone else would pick up the baton to flesh out crm.

cough @antaflos cough

I'm intensely interested in the added rules support.   My rules are pretty simple and I don't have a style preference, though it seems that gavmain's approach is a bit more flexible.   I run in a mixed Redhat/Debian environment and feature parity between crm and pcs is important to me.

I'd be game to look at crm support.
Paging @GiooDev - would you say that this PR is possibly superseded by #136?

I'd also be very interested in location rule support - at the moment, it's mostly for connectivity checks against a cloned ping resource.  In case it's of any help in implementing this for the crm provider, the following XML demonstrates a static preference and a rule preference on the same resource:
<constraints>
  <rsc_location id="ex_dummy-prefnode" rsc="ex_dummy" score="1000" node="lvs1"/>
  <rsc_location id="ex_dummy-prefping" rsc="ex_dummy">
    <rule score-attribute="pingd" id="ex_dummy-prefping-rule">
      <expression operation="defined" attribute="pingd" id="ex_dummy-prefping-expression"/>
    </rule>
  </rsc_location>
</constraints>

I expect the most difficult part would be in a full implementation of the various permitted expressions and combinations thereof.

Can you rebase and squash?

Can you rebase and squash? would love to have some tests as well (unit and integration)
thanks

I'll see if I can get time to come back to finishing it this week. There were a few questions from Felix regarding some of the code, some of which was lifted from other contributions to this module.

can you please add acceptance tests?

@gavmain this passes trusty. I would agree with @roidelapluie on acceptance tests.

hi @gavmain how is the current state here? Could you rebase and add the tests?

@bastelfreak you mean @GiooDev  :)

ups, yes.

@GiooDev requires rebase

Maybe I just don't understand enough about how the (pcs) provider is supposed to work. Is it correct that the self.element_to_hash method is never called? It looks like this is the key to parsing the rules from the XML output of the pcs command, right?
Anyway, I tried to get the type working for Debian. My changes to the crm provider: smoeding/puppet-corosync@975102b1c493a0062641a2e31c2e1c8af36dbb40 in the rule-cs_location-crm branch. Maybe this can be included somehow.

element_to_path is called here: https://github.com/voxpupuli/puppet-corosync/blob/bb34bf9048725ad61c3ea5d6afe73816bdf65e9a/lib/puppet/provider/cs_primitive/pcs.rb#L94 (lib/puppet/provider/cs_primitive/pcs.rb Line 94)

To me this looks like a different type using a different method of the same name.
In my XML the primitive elements are below the resources element. But rule elements are below the constraints element, so this is a different subtree. I still do not see how the rule elements are actually read and parsed by the pcs provider.

@GiooDev failing all the tests :(

How  can we mix this and #310?

see #356


defineProps with extended interface not working

Version
3.2.31
Reproduction link
stackblitz.com
Steps to reproduce
Create a component in SFC mode and use defineProps with an extended interface, properties are not working
<script setup lang="ts">
interface Example {
   description: string
}

interface Props extends Example {}

defineProps<Props>()
</script>

<template>
   <p>{{ description }}</p> <!-- the description is undefined -->
</template>

What is expected?
That the definition of properties have this support
What is actually happening?
defineProps with extended interface not working

Duplicate https://github.com/vuejs/core/issues/4498


query open state?

Hello,
great work
I have one question though.
Is it possible to query the open state?
I would like to turn on the light in the hall when it is dark outside and the door is opened.
Unfortunately I haven't found anything suitable yet.
Greetings from the "Nordsee"

Did you read the doc on the repo?

Here are those 3 sensors of the device:

Yes, I read the DOC in the repro. However, I only see the "unlocked" status. If I then open the door via the NUKI app, the status does not change.
Can you tell me what I have to write where?

I showed you two pictures, one is from the readme, one is the HA sensors, in the table above what do you read in description? Do you see the binary sensor with the description "shows the state of the door"? That's the sensor for the door. Then you have the binary sensor for the lock.

@lordnikon6
Hey,
Few questions though:

Do you have a (Nuki) Door sensor?
Do you have a Nuki Bridge?
When you say 'If I then open the door via the NUKI app', do you actually open the door or just unlatch the lock?

@alexdelprete
I think I found my mistake. I have a Nuki and a Bridge, but I don't have a separate Nuki door sensor.
I thought that if Nuki opens the lock for 3 seconds, this will be displayed briefly as a status. So the moment when the claw is pulled and the door opens.
Of course, I can also do this with a window or door contact. But then I would take an Aquara for 10 euros and not one from Nuki for 39 euros.
@kvj
my mistake. No, I don't have a Nuki door sensor. D
Yes, I have a bridge
And yes, I don't mean "unlock" I mean the moment the claw is pulled for 3 seconds and the door opens.

I thought that if Nuki opens the lock for 3 seconds, this will be displayed briefly as a status. So the moment when the claw is pulled and the door opens.

That's call UNLATCHING. The door status requires the sensor, that you don't have. :)

I can also do this with a window or door contact. But then I would take an Aquara for 10 euros and not one from Nuki for 39 euros.

sure, but you will have to integrate it manually in that case. :)

I don't mean "unlock" I mean the moment the claw is pulled for 3 seconds and the door opens.

you mean unlatching, see first reply of this post. :)

Thank you for your quick response.
I saw on the Nuki homepage that the door sensor is only suitable for Nuki 3 or 3Pro. I have the first generation Nuki. Unfortunately, the door sensor falls out for me :(
But now I know at least what it was.
Thank you for your help

Unfortunately, the door sensor falls out for me :(

Not a big issue: you'll save money with another sensor, and you can easily integrate it in HA.

+1 to using a more reliable zigbee/z-wave door sensor instead. Even though I have Nuki 2.0 with door sensor (which is just a square-shaped magnet), I rely on the zigbee one

Even though I have Nuki 2.0 with door sensor (which is just a square-shaped magnet), I rely on the zigbee one

Konstantin, could we support a generic non-nuki door sensor in the integration? So users could have a choice without having to integrate manually. We could ask in config_flow to optionally configure a binary door sensor, and that would replace the standard one.


k8s mutation library: auto-create prefix patches

Previously, kubernetes mutating admission control policies
could specify a JSON patch for any path where at most
the parent of the key being mutated was missing.
This change allows the mutating admission control policies
to specify any JSON patch for an object, and any additional patches
for a key's parent or grandparent or grandgrandparent,
etc. will be automatically created.
Note: this change does not handle patches for arrays

Very cool.


feat: fix ExecutorCompletionService workaround in ChunkProcessingPipeline

Contains
This pull request removes the work-around in ChunkProcessingPipeline where it reflects into the internals of the built-in ExecutorCompletionService class. This is no longer possible with newer Java versions and was never guarenteed to work in the first place.
Instead of making assumptions about ExecutorCompletionService, this pull request replaces the usage of the class with a new one specialised for dealing with chunks. Essentially, this makes sure that the submitted tasks are always of type PositionFuture, as several other classes in the codebase depend on the task future being an instance of that type. I believe that PositionFuture is used for certain optimisations related to the chunk load order.
This contributes to #3976.
How to test

Run the game
Create a new world
Run the newly created world
The world generation process should complete successfully
Chunks should load around the player in-game

Tested this out with Iota, world loaded quite the same as before this change, chunks appeared around the player, even after teleporting to a completely different location that was not generated before.


setSkin Method not working

Spawning the NPC works fine and everything but the skin is missing for some reason. I already tried with multiple skins, versions, and an added delay for showing the NPC.
No error messages and NPCLib loads correctly.
NMS Version: v1_8_R3
NPCLib Version: 2.0
    @EventHandler
public void onJoin(PlayerJoinEvent e) {
    MineSkinFetcher.fetchSkinFromIdAsync(725954, skin -> {
        NPC npc = npcLib.createNPC(Arrays.asList(ChatColor.WHITE + "test")).
                setLocation(e.getPlayer().getLocation()).
                setSkin(skin).
                create();

        Bukkit.getScheduler().runTask(this, () -> npc.show(e.getPlayer()));
    });

}

I think in v2.0 you have to set it via the texture and values.

Example, please?

I lied, but here is an exmaple:
https://github.com/JitseB/NPCLib/blob/master/examples/example-3.java

I tried that same thing :(

My mistake! I forgot to update the GameProfile after the skin object changed.
Building v2.0.1 right now. Will post the release shortly.

Release v2.0.1: https://github.com/JitseB/NPCLib/releases/tag/2.0.1


[WIP] Add Intersection Observer support to Prefetch component

Thank you for your hard work on react-static. I get goosebumps just experimenting with this masterpiece of a project.
This MR adds a new component PrefetchWhenSeen, which will fire the prefetch of its path when it is visible in the view. This opens up for powerful techniques like e.g. prefetching a page when the user first sees a link to it:
<PrefetchWhenSeen path={`/blog/post/${post.id}/`}>
  <Link to={`/blog/post/${post.id}/`}>{post.title}</Link>
</PrefetchWhenSeen>

Is this something you are interested in? The reason why I didn't put a new prop on the Prefetch component and add it conditionally that way, is because Prefetch has no wrapping element, so I could not put a ref on it.

Great work! This is what I like to see :)

Can you please open another PR and add some documentation to the README ? :)

Thanks! Absolutely. You would like the additions to the README in a separate MR?

Yes, since I preemptively merged this one already ;)


feature(app, api, robot-server): Reset gripper calibration and split OT-2 and OT-3 reset options

Overview
closes https://opentrons.atlassian.net/browse/RSS-232.
Add reset option for: Gripper Offset Calibrations .
Return the reset options per robot type.
Raise an error when trying to access an option that is not available on the robot.
Test Plan

Get reset options for an OT-2 GET {ROBOT_IP}/settings/reset/options and make sure we are only getting OT-2 options (covered under tavern tests and tested with dev server)
Get reset options for an OT-3 GET {ROBOT_IP}/settings/reset/options and make sure we are only getting OT-3 options (covered under tavern tests and tested with dev server)
POST a reset option that is not available on the OT-2 to the OT-2 and make sure you are getting a proper exception.
(covered with tavern test and test with dev server)

Changelog

Added a reset option for gripper_offset
Changed GET /settings/reset/options to return the list of options based on the robot_type.
Changed POST /settings/reset to raise an error if the option is not available on this robot.
Changed app side component to render gripper options from the server.

Review requests
changes make sense?
Risk assessment
low. should not change for OT-2. for OR-3 need to make sure reset gripper offset is working as expected.

It also looks like we have some tests timing out. Let's figure out what the deal is with those before merging this.

@TamarZanzouri
For DeviceResetSlideout, I can take care of it.
Actually I planned to work on that when the gripper reset option was ready.

this is what i'm seeing in the factory reset slide out when pushing to my flex dev kit — ill look into this tmrw:

@shlokamin
On my end the slideout worked as expected.
One quick question
Did you push this branch to your dev kit?
I just checked this branch the following command.
make -C robot-server dev OT_API_FF_enableOT3HardwareController="true" DEV_ROBOT_NAME="dev_server" OT_ROBOT_SERVER_simulator_configuration_file_path="simulators/test-flex.json"

@shlokamin @TamarZanzouri
with
make -C robot-server dev OT_API_FF_enableOT3HardwareController="true" DEV_ROBOT_NAME="dev_server" OT_ROBOT_SERVER_simulator_configuration_file_path="simulators/test-flex.json"

tried to push the branch to my dev kit and worked as expected.
Shlok, did you restart robot-server?
make push-ot3 host=my_dev_kit

lol my dev kit's robot server was just taking forever to respond, the reset options show up eventually. we don't have to worry about this now but we should hide the protocol run history and boot scripts sections unless we actually have those options to show


Update catch_human_agent.py with updated dm_env_adaptor.py behavior

An update to dm_env_adaptor.py (https://github.com/deepmind/dm_env_rpc/commit/7a06811544a8cb4ce3f460a2afa3bd4ed8e80a24#diff-2c755513ee4f5f32331e60c10ce8ef602679781e87629139b0ba03db85848ab0) changed the default behavior that this script relied upon. This script assumes the reward will be packed into the step_result.observation[_OBSERVATION_REWARD] as it was previously with dm_env_adaptor calls prior to the updated.
I believe that the current behavior requires that if you want the reward to be included into the observation you have to send it as an argument to dm_env_adaptor.py for example:
env, world_name = dm_env_adaptor.create_and_join_world( ..., requested_observations=[_OBSERVATION_BOARD, _OBSERVATION_REWARD])

The other way to get the reward is to call it from step_result.reward itself.
See pull request for one possible fix.

Thanks for reporting this! I don't see your PR, but this has subsequently been fixed this commit. Hopefully this clarifies how the dm_env_adaptor should be used.


internet_access:fee=no label is cut off, leading to misunderstanding

I was micromapping around this node today.
The internet_access:fee field is cut off, showing "Internet Access: No" in the GUI.

Updating the tag shows "Free"/no under "Internet Access ..." making it even harder to understand what's going on.

This might only be an issue with the smaller screen on an iPhone 12 mini which I'm using.

Alas it's not cut off: that is the exact label people see in e.g. iD editor. Source is here.
I recommend using the wifi field at the top part of fields: it allows setting both internet_access and internet_access:fee with a single tap.

Go Map!! pulls in the values and shows 'Free' instead on 'no', as you can see in the screenshots below.
Would it be possible to also show the verbose options instead of the value in Every Door?


LDAP Integration resources are missing

Hi, I would like to know if it's possible to add LDAP integration resources, specifically keycloak_ldap_user_federation and keycloak_ldap_group_mapper, which already exist for Terraform.
And also possibility of assigning keycloak role to LDAP groups.

pull request #94  will add this - i'll release most likely tomorrow

Release is out: https://marketplace.upbound.io/providers/crossplane-contrib/provider-keycloak/v0.20.0
Can you test and report back?
Also: Can you guys do me a favor and create some examples / docs for these resources?   I soon want to provide a big bunch of docs and examples / explanations

Thanks for the fast contribution! Your work is awesome. I’ll review and give feedback ASAP.

I can confirm that it's working properly! Thanks again @Breee

Nice, Could you Provide some realworld examples and docs for people that want to use the Ressources? (I don't have LDAP available without setting one up first)

Sure, do you want me to share examples here or add to examples folder in repo?

Here, I've added some examples I used to integrate LDAP with Keycloak for Kubernetes OIDC authentication. I'll include them in the examples here as well.
https://github.com/kaviankarimzadeh/k8s-keycloak-crossplane/tree/main/crossplane


[feature] Update project search fields on user full name change

This PR is an update for #1348, and should replace it.
Closes #1237.
Original author:
@huangginny
Original text:
Purpose
Originally, if a user changes his name and searches for his public projects, his public projects would be under his previous name, instead of the latest one.
Cause
Changing and saving the full name does not trigger a update on search on his projects, so the search fields that yield to the project remains unchanged.
Changes
Imported elasticsearch.helper. Now changing the full name will trigger the bulk api, so that the contributor list of the projects/components that this person is a contributor in will be updated in time.
Side effects
I haven't changed any existing code, so to my knowledge, nothing.

@lbanner, @GaryKriebel:
This PR lacks tests. Creating tests for it may be a suitable task for an intern, or it may not even be necessary.
@samchrisinger:
Could you look over this and let me know if anything jumps out to you? As far as I know, you're the current office Elasticsearch guru. :smile:

@lyndsysimon, @lbanner, @GaryKriebel this issue was reported again in #2565 and is also fixed by PR #2800, which is currently open. Probably best to consolidate the duplicate issues and choose a PR to go with.

sounds good, @samanehsan

@lyndsysimon and @samanehsan -- insight into which PR to make primary here? I'm good with you guys making that call. TY

I went ahead and closed the other PR - it was a single-line change, and did not address the performance concerns.

:+1:

I'm going to hold reviewing this code until asked by either @GaryKriebel or @lbanner

@sloria -- If you can review, maybe we tag this with 0.39.0 or 0.40.0? Thanks.

Tagged for 0.40.0. @fabianvf Can you please review this?

1st pass done :+1:

I just spoke with @fabianvf - since he's probably the most familiar with both the code and Elasticsearch, I've asked that he make the changes he's suggested and update this PR. I also made sure he has write permission to my fork so he can push his changes :)
Fabian said he'd probably get to it tomorrow morning. Once those changes are pushed, I'll take another brief look at it and send it your way @sloria.

@fabianvf - there are a lot more changed files here than I remember, including some that are not relevant to this PR (e.g., crud.py in the S3 addon).
Was there a merge conflict?

Hmm. There was a merge conflict, but only in elastic_search.py. Not sure why all this stuff is showing up as changed. Let me trying bringing it up to date with develop.

Looks great to me - @sloria, all yours now.


When a container has been removed, rewrite permissions back to user

What happened
After removing a site I attempted to clean-up the directory, but encountered a permission error when trying to remove the working directory.
What you expected to happen
I should have been able to shut down the site and then remove it without using sudo permissions.
How to reproduce it
Shut down a running site and remove it, then try to delete the directory the site was in. A permissions denied error should come back.
Related source links or issues
Command output
➜  drud-d8 ddev rm drud-d8
Stopping local-drud-d8-web ... done
Stopping local-drud-d8-db ... done
WARNING: Found orphan containers (local-drud-d7-web, local-drud-d7-db, nginx-proxy) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Removing local-drud-d8-web ... done
Removing local-drud-d8-db ... done
Network ddev_default is external, skipping
Successfully removed the drud-d8 application.
➜  drud-d8 ll
total 31504
drwxr-xr-x@ 23 csterling  staff   782B Jan 23 17:51 docroot
-rw-r--r--   1 csterling  staff   3.8M Jan 23 17:53 drud-d8.sql
-rw-r--r--   1 csterling  staff    12M Mar 24 13:28 production-drud-d8-1485219182.tar.gz
➜  drud-d8 cd ..

➜ rm -rf drud-d8
rm: drud-d8/docroot/sites/default/default.services.yml: Permission denied
rm: drud-d8/docroot/sites/default/default.settings.php: Permission denied
rm: drud-d8/docroot/sites/default/files: Permission denied
rm: drud-d8/docroot/sites/default/settings.php: Permission denied
rm: drud-d8/docroot/sites/default: Directory not empty
rm: drud-d8/docroot/sites: Directory not empty
rm: drud-d8/docroot: Directory not empty
rm: drud-d8: Directory not empty

➜ stat drud-d8/docroot/sites/default/default.services.yml
16777220 5818360 -rw-r--r-- 1 csterling staff 0 6762 "Dec 31 17:00:00 1969" "Jan 23 17:43:43 2017" "Mar 24 13:30:14 2017" "Jan 23 17:43:43 2017" 4096 16 0 drud-d8/docroot/sites/default/default.services.yml

➜ lsof drud-d8/docroot/sites/default/default.services.yml
<no output>

➜ id
uid=501(csterling) gid=20(staff) groups=20(staff),12(everyone),61(localaccounts),79(_appserverusr),80(admin),81(_appserveradm),98(_lpadmin),701(com.apple.sharepoint.group.1),33(_appstore),100(_lpoperator),204(_developer),395(com.apple.access_ftp),398(com.apple.access_screensharing),399(com.apple.access_ssh)

Closing this - it's a Drupal issue with sites/default, not a problem with permissions for the site files. Every user would have this exact same problem with any drupal install in any environment, including production. It's solved by chmod u+w sites/default


New Step Definition to select the export format

I could use an existing step definition.. Hence not required


Documentation about 'end-to-end' training is misleading

The example of model training in documentation is misleading, as it's not actually end-to-end - it trains detector and recognizer separately.
The example actually shows training of all parts of pipeline. The process isn't end-to-end though.

Thanks for raising this. I agree that the typical use of the term "end-to-end" in deep learning doesn't apply here and that the current wording is misleading. Would you be willing to file a PR to fix/change it?

Thanks for raising this. I agree that the typical use of the term "end-to-end" in deep learning doesn't apply here and that the current wording is misleading. Would you be willing to file a PR to fix/change it?


StreetView not showing

Google Maps Android API: The Google Maps Android API Street View service has been disabled for this application. This might be because of an incorrectly configured project, or an invalid API key. See the Google API console: https://console.developers.google.com, and information about Obtaining an API key: https://developers.google.com/maps/documentation/android-api/signup.

Can you clarify the issue you are encountering?


Remove Component derive for DepthOfFieldMode

Fixes #14592

This is technically a breaking change, but I'm not going to include it in the migration guide. There was no use for that trait impl!


Great library

Just wanted to say thanks for this library.  Just tried adding a file to IPFS, works great.
How complete would you say it is? is it like 90% of the IPFS API working?  Do you plan on adding any new features?
I'm going to ask the IPFS guys to link to this repo for a C# implementation if you don't mind,.

Hi @slothbag. Thanks very much.
I think it's almost there. I've just checked the command reference and some new commands have popped up since I last checked, so I might look into adding those before merging into master & tagging. It'll probably need some clean-up before I'm 100% happy with it, then I'll look into publishing a nuget package to make it super easy to use.
I'll do some more testing as soon as I have time. If you find anything that doesn't work as expected then I'll get onto it asap.
-TD

The tuple<string, stream> syntax was confusing at first.. I had to dig through source code to realize the string was for the filename :)
nuget would be awesome.. my latest C#/JS project I have been packaging everything using the IPFS based GX package manager.. i'll add this too :)

Ha, can you believe it, my project requires a IPFS API library and an Ethereum API library.. I just saw your other repo :)
I have been using https://github.com/Nethereum/Nethereum but its still very much alpha right now.  How stable is your lib?

Ah, yeah, sorry about that tuple syntax. That's the first thing on my list to change :)
I'm using both the Ipfs/ethereum clients I wrote for a work project right now, and haven't had many problems recently. I obviously fix any if/when we come across them, but because we mostly use the same functions a lot of the time there are some features which aren't as tested as I'd like them to be.

Ha that's lucky then. Sorry about that tuple syntax. That's the first thing on my list of changes :)
As for stability: I'm using both ipfs/ethereum clients for a work project, so they'll be supported & improved upon for a long time (at least a couple of years). Any problems we come across obviously get fixed asap, but because we use the same function a lot there are some things that aren't as tested as I'd like.
Nethereum looks good, a bit complex/bulky for my liking though.

@slothbag Tuples are now NamedStreams for IpfsClient.Add(...) :)


Updated chat-pdf-huggingface to easily install all requirements in Python 3.10.

I'm learning and enhancing my NLP/GenAI skills and am impressed with your project and am hoping to learn more from it.
I had issues install the requirements.txt, thus I removed the '==version' in requirements.txt with sed -i 's/\=.*//g' requirements.txt.
I added some modules that are needed for app.py that were not in requirements.txt.
I removed pywin32 and pywinpty from requirements.txt, since I'm using Ubuntu 22.04.
I added MIT LICENSE.
I hope this pull request can be helpful to you. Feel free to disregard if not.

I also removed .env.example, and added README.md instruction to create .env with the token, which will be ignored by .gitignore


[Question] How to execute a form with a command?

I know it´s possible a dumb question but i´m newbie i just created like one custom command, so i created a form but i don´t know how to trigger it because there isn´t something like "edit trigger" in the form creation, i created this form

And i want to execute it with a slapper or a sign, How to trigger a form?

To add trigger, you have to create a recipe and use Form > send a form action. and then you will find a Edit trigger button on recipe menu.

To add trigger, you have to create a recipe and use Form > send a form action. and then you will find a Edit trigger button on recipe menu.

Tysm, i warned that was probably a dumb question xD

Now i have slappers in a shop and when i touch them the form appears


Integrate External Secrets Operator

Instead of supporting encrypted secrets stored in Git, Declcd should integrate "External Secrets" (https://external-secrets.io/latest/)

Tested and works. No integration needed, only documentation and howtos/recommendations


Verify column size for phase

It seems wrong in https://github.com/jcgay/buildplan-maven-plugin/blob/9937ea6774f546d5c407123d200c38db369b81cc/src/main/java/fr/jcgay/maven/plugin/buildplan/display/AbstractTableDescriptor.java#L48
It should only read size from execution.getLifecyclePhase().

🤷♂


Query : Aws elastic search plugin will support multiple outputs(i.e s3)

Problem
Unable to send logs to AWS ES and S3 at same time and nothing shows up in logs. At this point of time would like to check AWS ES plugin will support s3?
Td-agent version : 2.5.x
Infrastructure > AWS ECS > Fluentd Aggregrators (2 instances and load balanced) > ES > Kibana
As we see some lags (5 to 6mins) in logs processing would like to check with adding other output(like s3) to view the logs. Is this right action to take?
Config :

        @type forward
        @id input_forward

<filter *.**>
@type record_transformer

container_id ${record["container_id"]}

<match *.**>
type aws-elasticsearch-service
logstash_format true
logstash_prefix ${tag[0]}
include_tag_key true
tag_key "Application_name"
flush_interval 5s
<buffer>
  flush_at_shutdown true
  flush_mode immediate
  flush_thread_count 8
  flush_thread_interval 1
  flush_thread_burst_interval 1
  retry_forever true
  retry_type exponential_backoff
</buffer>

    <endpoint>
            url https://vpc-xxxx.aws.amazon.com
            region us-east-1
            access_key_id "xxxxxxxxxxxxxxxx"
            secret_access_key "xxxxxxxxxxxxxxxxxxxx"
    </endpoint>

OS version - Ubuntu 18.04 LTS
td-agent version : 2.5
AWS ES version - 6.8
fluent-plugin-aws-elasticsearch-service (2.1.0)

Hello,
Its been awhile since i've worked with this plug-in and I remember having a similar issue. Take a look at the copy plugin https://docs.fluentd.org/output/copy. This is mostly likely what your looking for.
Jason

Yeah, @jayslife answer is right.
Output plugin should emit nothing.
Instead, we can reprecate Fluentd events with copy plugin.

Thanks for the reply!
I tried with Copy plugin but not working. Could you give us any example here that will be great!


Continuation of redesign.

Reopening old Pull Request and previous task.

Believe work is complete. In-sampling added another dimension of complexity.

Handling Unit Tests that are failing on Python3.8 for an unknown reason. Previously passed and merged into Git.


Cluster alerts are only visible to cluster members

Rancher versions:
rancher/server or rancher/rancher: v2.0.4
rancher/agent or rancher/rancher-agent: v2.0.4
We'd like to create read-only monitoring users for Rancher. We're able to grant the "View Nodes" permission to a user and scrape cluster component health and node health states.
However, configured cluster alerts are only visible to cluster members. It'd be nice if there was a "View Cluster Alerts" permission, preserving read-only access to the cluster.

I just found out ClusterAlerts are stored on the Rancher Cluster itself. So I created a custom role with GET / LIST / WATCH on ClusterAlerts and assigned it to to the monitoring user on the local cluster. This works fine.


NestJs Integration - InternalOAuthError

Hi,
I am struggling to integrate this passport strategy with NestJs. I have tried  a number of different ways, from using the official AuthGuard and strategy class to calling passport.use / passport.authenticate directly with the same strategy instance.
Here is my strategy class:
import { PassportStrategy } from '@nestjs/passport';
import * as TwitterStrategy from '@superfaceai/passport-twitter-oauth2';
...

@Injectable()
export class TenantTwitterAuthStrategy extends PassportStrategy(TwitterStrategy, 'twitter') {
    private readonly configService: ConfigService;

    constructor(config: ConfigService) {
        super({
            clientID: config.get('TWITTER_CLIENT_ID'),
            clientSecret: config.get('TWITTER_CLIENT_SECRET'),
            callbackURL: `${config.get('HOST')}/auth/tenant/twitter/redirect`,
            clientType: 'private',
            scope: ['tweet.read', 'offline.access'],
            passReqToCallback: true,
        });
        this.configService = config;
    }

    public validate(req: IAuthedRequest, accessToken: string, refreshToken: string, profile: TwitterStrategy.Profile) {

        ....
     }

I have attempted to debug, and I can see the the userProfile function is getting what looks like a valid accessToken, and the url object is:
{
  href: 'https://api.twitter.com/2/users/me?user.fields=profile_image_url,url',
  origin: 'https://api.twitter.com',
  protocol: 'https:',
  username: '',
  password: '',
  host: 'api.twitter.com',
  hostname: 'api.twitter.com',
  port: '',
  pathname: '/2/users/me',
  search: '?user.fields=profile_image_url,url',
  searchParams: URLSearchParams { 'user.fields' => 'profile_image_url,url' },
  hash: ''
}

which looks correct. However it's throwing an InternalOAuthError: Failed to fetch user profile.
Which is a result of a 403 Forbidden response coming back from the profile call.
Has anyone managed to sucessfully integrate this with NestJs. I haven't had issues with other providers or indeed the original passport-twitter library, however I am wanting to use the v2 API.

Hi, looks like problem with scopes. Try to add users.read scope to list of requested scopes in the strategy constructor.

Ah, gotcha, missed that one which is easily done. I raised a PR to have that scope as a minimum default for the strategy to work:
https://github.com/superfaceai/passport-twitter-oauth2/pull/11

Released as v1.1.0


HTTP error! Status: 504

> cloudflare-delete:list
> node cf_list_delete.js

file:///home/runner/work/cloudflare-gateway-adblock/cloudflare-gateway-adblock/lib/helpers.js:40
    throw new Error(`HTTP error! Status: ${response.status}`);
          ^

Error: HTTP error! Status: 504
    at request (file:///home/runner/work/cloudflare-gateway-adblock/cloudflare-gateway-adblock/lib/helpers.js:40:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async file:///home/runner/work/cloudflare-gateway-adblock/cloudflare-gateway-adblock/cf_list_delete.js:9:29

Hi, my github action run failed and show up this HTTP error message, it works great before.
Is there any fixable problem? What info should I provide for maintainers?

Addition info is that I found out my Zero Trust team/lists webpage takes like forever to loading, is that the reason that cause 504 status code?

Cloudflare's core data center is offline due to commercial power outage and the generators failed.  Based on the latest update from cloudflarestatus.com , it says that all services has been restored but noticed in their previous update that the List (updating of list items) was temporarily suspended. It is not clear if this is the reason for error 504 since all other service such as Cloudflare API, ZT Gateway are already restored.

Just to add, I have other script that will read the IP address from NO-IP and push that IP into the DNS Location in my Cloudflare Gateway account and it is now working properly. I suspect that the suspension of List (updating of list item) that was mentioned on the Cloudflare's advisory is what causing the CGPS script to fail with error 504.
Their advisory is confusing. First, they suspend the List (updating of list item) and then they provided a new update saying that "All services" has been restored and that they are now monitoring the stability.  I think they forgot to restore the List? 🤔

As @OneZK said, Cloudflare is experiencing issues on their side and this is not a problem with this project. See the Cloudflare Status for details. I'll keep this open just in case other people come across this.

Thanks for contributors' reply, I am fully understand currently status.
I'll keep re-execute my github action to see if all go back to normal, if so, then close this issue.
👍

According to the cloudflare status page here: https://www.cloudflarestatus.com/incidents/hm7491k53ppg

Material changes since last update: Lists have now have been moved to restored status.  Lists can only be modified through the Dashboard currently.
Posted 13 hours ago. Nov 04, 2023 - 18:32 UTC

So, the lists should be accessible from the dashboard. But even after 13 hours now, it's still taking forever to load the lists page on the dashboard.
And on top of that, the lists loads once or twice after waiting for like several minutes and it shows that there are no lists.
I had a few other lists apart from this project, and they are gone now. Hope it gets recovered from some backup that cloudflare has in a few days.
I have raised a support ticket to cloudflare.

According to the cloudflare status page here: https://www.cloudflarestatus.com/incidents/hm7491k53ppg

Material changes since last update: Lists have now have been moved to restored status.  Lists can only be modified through the Dashboard currently.
Posted 13 hours ago. Nov 04, 2023 - 18:32 UTC

So, the lists should be accessible from the dashboard. But even after 13 hours now, it's still taking forever to load the lists page on the dashboard. And on top of that, the lists loads once or twice after waiting for like several minutes and it shows that there are no lists. I had a few other lists apart from this project, and they are gone now. Hope it gets recovered from some backup that cloudflare has in a few days. I have raised a support ticket to cloudflare.

After investigating this through github and cloudflare side, I can confirm that cloudflare gateway was able to receive the domains pushed by this script based on the logs and it seems that all chunks are successfully pushed into CF gateway even though the cgps action logs shows an error on all the chunks. It sames like the CF Gateway is unable to create a Rule/Policy. Now another problem is that those chunks of domain that was pushed into cloudflare gateway wont appear when we manually create a policy with the following selector
-> Domain
-> In list
-> No Valid Options
It shows No Valid Options so it looks like those chunks becomes invisible in dashboard and through API. CGPS script cannot find those chunks when it runs the delete request to the CF API.

Last status from Cloudflare concerning lists

Just testing now. Looks like everything is working again.

I'm still getting some unexplainable authentication issues.

My github action works great now. Thanks for tracing this issue.
https://www.cloudflarestatus.com/

I think there's still a problem with  Cloudflare  API_TOKEN when FAST_MODE is Enabled. I forked a CGPS repo from someone who is using API_TOKEN instead of API_KEY. I got HTTP Error 504, 500 and 409  with FAST_MODE enabled but when FAST_MODE is disabled, everything works fine.  On the other hand, the CGPS repo that I forked directly from @mrrfv which us using API_KEY works great on both FAST_MODE enabled and disabled.
I'll investigate this issue when I have time.

I think there's still a problem with Cloudflare API_TOKEN when FAST_MODE is Enabled.

Is there any update for this situation? Just curious

I think there's still a problem with Cloudflare API_TOKEN when FAST_MODE is Enabled.

Is there any update for this situation? Just curious

It works fine now. I ran the script with FAST_MODE enabled and it succeded without error and all chunks are created into gateway policy successfully and the old list in gateway has been deleted without error.

Okay, then I will close this issue. Thanks for everyone's help. :)


Add knapsack exercise

I would not require an Item class. If the student wants to create one, that's their choice.
I'd like to see the tests use arrays of maps.
    var items = [
      {"weight": 100, "value": 1),
    ]

Tests now use arrays of maps.

Thanks!


Thanks for sharing the code. Would you develop a sample client app, for example angualr/react/vue, to consume the api?

It's not in our plans at the moment, but it may be in the future.


Support bacon-qr-code 2.0.0

Add support of bacon-qr-code v2.0.0
Class Png does not exist anymore.
We have to use ImagickImageBackEnd instead - which require imagick php extension.
I've also added the support of svg image, by adding a new parameter to getQRCodeInline.
PS: the test on line https://github.com/antonioribeiro/google2fa/blob/master/tests/Google2FATest.php#L220 does not work on my computer: I hav php 7.1, and I get QRCODEPHPABOVE72 result...

Ok I did not saw the aliasing is very super bad with the new version of bacon-qr-code:

... and the image size (base64 encoded) is larger!

People are not unanimous about Bacon, many have asked to remove it from this package, so now that we are having to add many references to other Bacon classes, maybe it's time to make it, somehow, an optional QRCode plugin.

The change also breaks compatibility with PHP 5.4:

1) PragmaRX\Google2FA\Tests\Google2FATest::testQrcodeInline
BaconQrCode\Exception\RuntimeException: You need to install the imagick extension to use this back end

This is probably a reason to remove it completely from this package.

"This is probably a reason to remove it completely from this package." totaly agree !

@asbiin that anti-aliased version surely doesn't look that nice. That was basically worked around in the old renderer but automatically adding a few pixels of padding around the QR code to result in an exact pixel size per module. This was considered a bug by some people though, which is why this was added.
An option to solve this would be a setting on the Imagick renderer to opt-in to add optional padding if required, so that each module is pixel-perfect.
As for the requirement on the imagick extension. What phpMyAdmin is doing to solve that (and also get rid of the gd extension requirement in version 1) is to simply use SVG. There is practically no browser left without SVG support, so that is a vital solution.


5.0.0 fresh install yields no template

fresh install with no other dependencies yields
A template was not provided. This is likely because you're using an outdated version of create-react-app.
node v16.13.2
npm v8.1.2
relevant package.json:
  "dependencies": {
    "create-react-app": "^5.0.0"
  }
}

no global installs

found the issue there was a local outdated install in usr/local/bin/create-react-app. simply rm that file and you're good to go.
reference: https://stackoverflow.com/questions/59188624/template-not-provided-using-create-react-app


chore: add translation for back to quote CTA in WebPTXPlayer

✅ Checklist

[x] npx changeset was attached.
[x] Covered by automatic tests.
[x] Impact of the changes:

...

📝 Description
Replacing a hardcoded value for "Quote" with new common translations in LLM and LLD, in WebPTXPlayer.

❓ Context

JIRA or GitHub link: LIVE-9270

🧐 Checklist for the PR Reviewers

The code aligns with the requirements described in the linked JIRA or GitHub issue.
The PR description clearly documents the changes made and explains any technical trade-offs or design decisions.
There are no undocumented trade-offs, technical debt, or maintainability issues.
The PR has been tested thoroughly, and any potential edge cases have been considered and handled.
Any new dependencies have been justified and documented.
Performance considerations have been taken into account. (changes have been profiled or benchmarked if necessary)

LGTM! Great work!


iOS top task bar is white background and white icons when flutter appbar is white.

If I create a scaffold like below,
new Scaffold(
        appBar: new AppBar(
          leading: new BackButton(color: Colors.grey[600]),
          title: _buildSearchBar,
          backgroundColor: Colors.white,
        ),
        backgroundColor: Colors.white,
        body: body);

In Android, the search bar is white and the top task bar (a thin bar at very top showing time, battery and icons) is grey.
In iOS, the search bar is white but and top task bar is white too. It masks all the white icons and timestamp so we can't see anything.

@Hixie What do you think about this? I would imagine having a search bar in white is quite common.

(Side note: when changing this kind of thing I would recommend just wrapping the AppBar in an appropriate Theme so that you don't have to individually control each component.)
This is a dupe of https://github.com/flutter/flutter/issues/4630; in the meantime you can work around it by manually calling SystemChrome.setSystemUIOverlayStyle().

Hi Hixie,
I tried SystemChrome.setSystemUIOverlayStyle(SystemUiOverlayStyle.light);, but it doesn't work. That's because this method also get called inside the appbar build method.
The theme of the status bar is always the same as the theme of the AppBar. So it works with manually setting the app bar brightness to Brightness.light when build a single Scaffold widget with the current primaryColorBrightness == Brightness.dark. In this case the system icons (carrier, battery and etc.) are in black.
But when this Scaffold widget(2) is in the MaterialPageRoute builder and pushed from another Scaffold widget(1) with Theme.primaryColorBrightness == Brightness.dark, it doesn't work for the first build of wiget (2). It works when widget(2) get a rebuild.
I just filed a bug related to this issue(https://github.com/flutter/flutter/issues/12960) :]

@jonahwilliams I'm closing as fixed by #17672. If that's not right, please reopen.


Does not work anymore

App outlet does not work anymore (does not fetch any content). Also, all links on the website that point to Heroku resources are dead.
I am using App Outlet 1.2.0 (appimage).

duplicated of #70


User account for vmware_exporter

Hi,
Could we use a separate user account to scrape these metrics instead of administrator. If yes, please let me know how to create one?
TIA

Yes, you can, but this is related to vsphere's permissions, not to vmware_exporter. I don't remember the details, but if you read a little bit about it, you'll understand it eventually. You could start here: https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.security.doc/GUID-5372F580-5C23-4E9C-8A4E-EF1B4DD9033E.html


vim-airline should set gui colors regardless of &termguicolors

Hi,
vim-airline currently has an airline#init#gui_mode function which it uses in order to decide whether gui= colors should be set. I believe vim-airline should set gui colors unconditionally instead.
Indeed airline#init#gui_mode does not properly detect if a GUI is present. gui_running does not seem to be a standard variable (:helpgrep gui_running does not return anything in vim or neovim). It is not set by Vim, GVim, Neovim or Neovim UIs.
&termguicolors does not indicate whether vim/neovim is running in a gui either, as described in the documentation:
Vim:

When on, uses |highlight-guifg| and |highlight-guibg| attributes in
the terminal (thus using 24-bit color).
Requires a ISO-8613-3 compatible terminal.  If setting this option
does not work (produces a colorless UI) reading |xterm-true-color|
might help.
For Win32 console, Windows 10 version 1703 (Creators Update) or later
is required. Use this check to find out: >
if has('vcon')
<	This requires Vim to be built with the |+vtp| feature.
Note that the "cterm" attributes are still used, not the "gui" ones.
NOTE: This option is reset when 'compatible' is set.

Neovim:

Enables 24-bit RGB color in the |TUI|.  Uses "gui" |:highlight|
attributes instead of "cterm" attributes. |highlight-guifg|
Requires an ISO-8613-3 compatible terminal.

Both descriptions are very specific about termguicolors only indicating that vim/neovim is running in a terminal and that the terminal supports 24-bit colors. Neither GVim nor most Neovim UIs set this setting.
Refusing to either set gui_running or termguicolors might look like pedantry at first but it isn't: Neovim's UI protocol sends both gui colors and cterm colors to the UI and lets the UI decide what information it wants to use. For example, neovim-pytc-example uses the cterm colors while firenvim uses the gui ones. This allows multiple GUIs to connect to a same neovim instance and still have the expected colors in both UIs. Something that can't happen if vim-airline only sets either cterm or gui colors.
Is there anything preventing vim-airline from setting both cterm and gui colors? I'd be happy to discuss that and try to find a solution.

Yes, airline#init#gui_mode is specifically there to decide, whether we need to use gui attributes for the highlighting groups or cterm attributes. While we could also use the cterm attributes, this of course has some runtime costs, that I like to avoid (and it makes computation of the right attributes a bit easier).
I am not sure, why you say :helpgrep gui_running does not get you any result, it should give you several, most importantly this one below :h feature-list

gui_running             Vim is running in the GUI, or it will start soon.

So the important thing is, when a gui is running or termguicolors is set, Vim will use the gui related attributes for highlighting and we remember this setting. So I am not sure what you want me to do here.

While we could also use the cterm attributes, this of course has some runtime costs

Are there benchmarks somewhere that I could use to profile and understand how adding cterm colors affects vim-airline's performance? Perhaps we could find other optimizations that would make this cost acceptable while still allowing cterm colors.

it makes computation of the right attributes a bit easier

I don't know vim-airline's code well enough to understand this. Could you expand on that? (Just let me know if you don't have time, I'll try to dig more into the code base to get a better understanding.)

I am not sure, why you say :helpgrep gui_running

Ah, you're right! It turns out that Debian symlinks vim to neovim, so I was actually running the command in Neovim twice instead of once in Vim and once in Neovim, sorry about this.

I don't know vim-airline's code well enough to understand this. Could you expand on that?

In the past there were many complaints about performance and one of the main performance issues was re-defining highlighting groups dynamically. I think :hi commands are kind of expansive, so therefore I am not fond of adding more attributes.
However, I am still missing why you need cterm color attributes. So what problem are you facing?

However, I am still missing why you need cterm color attributes. So what problem are you facing?

The problem I am facing is that people are trying to use my Neovim UI and vim-airline at the same time :).
Neovim's UI protocol sends both cterm and gui colors to the UI. This because multiple, different UIs can connect to the same Neovim instance and while some might care about the GUI attributes, others might care about cterm. Currently, if vim-airline thinks it's in a terminal, the gui colors sent by Neovim will be wrong. If vim-airline thinks it's in a gui, the cterm colors will be wrong. There is no way to have vim-airline send the right colors for both cterm and gui at the same time.

This because multiple, different UIs can connect to the same Neovim instance and while some might care about the GUI attributes, others might care about cterm.

I have heared that claim since the beginning of Neovim (and remember some discussion about it on the neovim issue tracker). But are people really use the same neovim instance with several different UI? Why would you edit the same files in different UIs at the same time? I mean, I can easily imagine doing two different conflicting things in those different UIs, that would be a much greater problem than different highlighting groups.

If vim-airline thinks it's in a gui, the cterm colors will be wrong. There is no way to have vim-airline send the right colors for both cterm and gui at the same time.

So why does your UI need both colors? I suppose the UI will either support RGB colors (termguicolor setting) or use terminal like coloring. So why does it need to care about colors that won't be shown anyhow?
Note, I am trying to understand the problem. Perhaps this needs some adjustments in vim-airline to support different UIs in neovim at the same time better, even if I think this would be a rather niche problem here.

But are people really use the same neovim instance with several different UI?

I do actually :). Right now it's only for debugging, but I do hope to do this for real some day: my UI runs in the browser and I'd like it to connect to the same neovim server I'm connected to in my terminal. I hear it's not that uncommon to connect multiple clients to a same server in the emacs community.

Why would you edit the same files in different UIs at the same time? I mean, I can easily imagine doing two different conflicting things in those different UIs, that would be a much greater problem than different highlighting groups.

I don't see how you could do conflicting things with multiple UIs connected to the same neovim instance. Since there's only one instance, the file is open only once, you're just looking at it from a different X window.

So why does your UI need both colors?

It doesn't. It only needs gui colors. But my terminal needs cterm colors. If both are connected at the same time to the same instance there will be a conflict.

Note, I am trying to understand the problem. Perhaps this needs some adjustments in vim-airline to support different UIs in neovim at the same time better, even if I think this would be a rather niche problem here.

I completely understand, I also push back when people ask me to implement features in my UI :). I'll try to benchmark vim-airline and the :hi command this week-end and see if there are low hanging fruits wrt optimizations.

A plugin should preferably not depend on the value of &termguicolors at startup/initiation. This value can be changed by either the user (both in vim and neovim) or by external UI (in case of neovim). I would be surprised if setting half of the color attributes saves any significant start up cost, unless airline defines ten of thousands of highlight groups.

I don't agree for various reasons. First of all ternguicolors is probably a setting, that will be set in the initialization file and probably won't be changed later. So once the plugin loads the value can be trusted. However it can be changed later and for that case an OptionSet autocommand exists, the same is true for starting the gui later.
Second  of all, it is perfectly valid to have highlighting groups that only contain some attributes. There is nothing in the documentation that mentions this should not be done. In fact for such a case, perhaps it's the externals UI responsibility to transform the terminal only attributes to gui attributes.
But as said before, I am not completely against adding some code to handle this better for neovim. But I have to check the highlighting code but my feeling is, it should not be too hard. Let me check later today.

Am 26.10.2020 um 16:30 schrieb Björn Linse<EMAIL_ADDRESS>
A plugin should preferably not depend on the value of &termguicolors at startup/initiation. This value can be changed by either the user (both in vim and neovim) or by external UI (in case of neovim). I would be surprised if setting half of the color attributes saves any significant start up cost, unless airline defines ten of thousands of highlight groups.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.

Second  of all, it is perfectly valid to have highlighting groups that only contain some attributes. There is nothing in the documentation that mentions this should not be done. In fact for such a case, perhaps it's the externals UI responsibility to transform the terminal only attributes to gui attributes.

Anything a plugin does is perfectly valid by definition (as there is no spec for what a plugin should do). The point is just that a plugin that simply defines all the attributes already will give a more painless and smooth experience to end users, at the cost of a few extra microseconds at startup. In my limited experience.

Well yes, but the point being, that Vim was not really designed for handling several guis that handle different highlighting attributes. So for Vim the existing behaviour is perfectly valid and fine (and is already painless and smooth behaviour).
Anyhow, please check the  refactor_airline_gui_mode branch

So for Vim the existing behaviour is perfectly valid and fine (and is already painless and smooth behaviour).

Do whatever you want. But be aware that dynamic termguicolors has existed in Vim for multiple years, nothing about this issue is really Neovim specific.

But be aware that dynamic termguicolors has existed in Vim for multiple years, nothing about this issue is really Neovim specific.

As I have explained, vim-airline took care of changing termguicolors setting already.

I'm pretty sure it is not. At least, for all the cases I am aware of (which is quite a few), I have specifically changed the behaviour of rgb colors in neovim to match the ones of vim (both w.r.t changing tgc and switching between term and gui). You reasoning seems to be if you haven't personally seen an issue in vim, and a neovim user happens to break the news, it must be neovim specific by default.

Okay, at the risk of becoming offtopic, let me further explain:
I know of two cases, where vim changes what highlighting attributes are actually used. The one is switching on termguicolors setting, which makes the terminal use the gui[fb]g attributes. This can be switched on/off by using the termguicolor setting. So switchting that option should re-initialize airline#init#gui_mode. For that specific case, the OptionSet autocommand exists:
https://github.com/vim-airline/vim-airline/blob/8608270bd39e9aa31bbdb8cd22b2ba776037feb6/plugin/airline.vim#L143-L146
That should work in both Neovim and Vim. The other option is to use the :gui command, which I believe does not exist in Neovim, so is no problem there. For that event, the GuiEnter autocommand exists in Vim:
https://github.com/vim-airline/vim-airline/blob/8608270bd39e9aa31bbdb8cd22b2ba776037feb6/plugin/airline.vim#L142
I am not aware of other possibilities to switch what attributes are actually used (and I haven't heard complaints). Yes, you can use :noa :set or have a plugin change termguicolors setting after vim-airline was loaded. But at least until now, this has seemed to work well. And if a user decides to use :set eventignore=all and wonders why vim-airline doesn't change colors, I would not consider this a bug.
If you know of other cases where Vim changes what attributes are used for highlighting, please let me know and we can discuss how to fix this.

You reasoning seems to be if you haven't personally seen an issue in vim, and a neovim user happens to break the news, it must be neovim specific by default.

because I haven't seen the problem we are talking here in Vim and even for Neovim users, this seems to be a rather niche use case (using several different UIs with the same server). And yes, I have seen too many requests from Neovim users that something does not work the same as in Vim. This has caused me personally lot of debugging and maintenance issues in several of my plugins and I still remember the Neovim will stay compatible with VimScript at least 90% of the features. That claim does not seem to be valid anymore.

I have specifically changed the behaviour of rgb colors in neovim

Yes and I have fixed several issues in Vim (for example I contributed the mentioned OptionSet autocommand, but I have also changed some color issues after merging the termguicolor patch) from which Neovim also benefits. So what?
You make it sound like I am actively against Neovim. I am not. I just follow the Vim features more closely. And If I get aware of Neovim problems, I try to fix it if I can (or happily take patches). To come back to the issue here, I have just changed the handling of how highlighting colors are used in the mentioned refactor_airline_gui_mode branch. Please test that this works correctly and I will merge that one.

@chrisbra The refactor_airline_gui_mode branch fixes the issue I was seeing, thanks a lot! Do you know what the performance impact is? If not, do you want me to measure it?

Let me just merge it and hope that this won't impact performance negatively. There are a couple of options that should help in case it does.

Let's not waste each other time much more, but:

But at least until now, this has seemed to work well. And if a user decides to use :set eventignore=all and wonders why vim-airline doesn't change colors, I would not consider this a bug.

This is the thing I was getting at. The relevant underlying design desicion, support for multiple simultaneous color formats (which if not respected can manifest in bugs in more or less niche usecases, in various vim and neovim versions and patch levels) was known to exist in early versions of vim, but yet this implies  "Neovim will stay compatible with VimScript at least 90% of the features. that claim does not seem to be valid anymore."
Neovim aims to make it easy for a "well designed" vim plugin to support neovim without too many if("nvim") (or at very least, you can just hang on the has("nvim") to the already necessary vim-patch check), but bug-for-bug compatibility with all autocmd quirks is not our goal. But as there already are many such vim+neovim plugins that seems to agree with my/our particular idea of being compatible or "well designed"  (lightline has been working very smoothly with all my most insane floating window hacks, even without any nvim specific fixes or workarounds) and which we can recommend to our users instead, I don't need to waste your time further if you do not want to.

Please do not assume that vim-airline is not well-desigend. If you want to discuss certain aspects that you think vim-airline does not behave corretly fine, I am all for a technical discussion. But if you do not know vim-airline, then please leave out such assumptions. FWIW, vim-airline just has a couple of has("nvim") (mostly for the terminal extension and the async api) and as I said, if I here of problems with Neovim I will try to fix it (as this issue here shows).


Chunk with polyfills is not added to the html file

I decided to try this tool in Windows. I ran the installation, then build the project preact build. After that, the browser opened the Internet explorer 11, and saw 'Promise is undefined' in console.
In our readme it is indicated that polyfills for promises have been added, but I do not see them in the resulting html-file.

Is it a bug, or have I not finished reading the documentation?

In the current npm version of preact-cli, the polyfills are loaded using a dynamically created script tag, not directly in the html.

@jt3k this will be fixed in next release. What @developit described is future behaviour.
Right now pollyfils are placed in weback generated chunk, which is loaded by weback at runtime - however this requires a Promise to exist in the first place and fails on IE.

ок! thanx


feature request: add option to generate TOC for pdf output

Since now headers and footers with page numbers work, I now desperately miss an option to generate a Table of Contents (TOC) out of the h1- h7 headers when generating a pdf file (i.e like wkhtmltopdf is doing this). The TOC should be at the start of the pdf and it should not only be clickable (jump to the page) but also generate the outline pdf element, so that the TOC is displayed in the contents view in any viewer. Although this may sound complicated if this functionality is implemented at the right place it is not that complicated (take a look on how wkhtmltopdf is implementing this).
Before posting here I tried a couple of workarounds to achieve this.
Some dead ends:

CSS3 target_counter() -> proposed a long time ago and only some specialised tools do it, to be honest I've given up to think that it will be implemented in chrome some day (reference issue is now: https://bugs.chromium.org/p/chromium/issues/detail?id=368053 )
Find a tool or tools to extract the table of contents from the generated pdf and generate a preface.pdf with the TOC witch to merge in-front of the original pdf ... With https://github.com/qpdf/qpdf I was able to generate a readable and searchable "pdf text-file" so that theoretically it was possible to find the header in the text file and via reverse search and the added comments find out on wich page it is etc. etc...

Any chance to get this soon?
Thanks
Ognian

Hi @Ognian  and @aslushnikov ,
I had a similar TOC problem in another context, and I solved it using tool wkhtmltopdf.
This tool implements a few good ideas, which you can get inspired from:

Options --outline, --no-outline and --outline-depth <level> set the PDF outline (better called the document outline dictionary)
The table of content can be inserted using the default stylesheet, or a user-provided one.
I thought, this was unpractical, instead I used option --dump-outline to create an external file containing the headers page numbers, and converted it into a JSON object, which is then inserted into the HTML document on the next read (TOC is then created in JS). This requires generating the document twice to get the page numbers right, but I see no better algorithm.

Therefore, my first proposals for extending page.pdf():

add option outlineDepth: defaults to 0 (meaning no outline in PDF), else a value like 4 includes all
 to
 into the PDF document outline;
eventually add option outlineExcludeSelector: if element matches selector, then it is not included into the generated outline.
For example with value 'h3.no-pdf-outline', any
...
 would be excluded from the outline.

As for the table of contents with page numbers inside of the document : this is part of a more generic problem.
For example, users may want a table of figures, a table of tables, or a keywords index. And these different tables may be inserted at different places into the document : there are books with a table of contents at the end, or at beginning of each chapter. All of this depends on the editor, the cultural context, etc...
Therefore, I think that generating a list of page numbers for all the specified DOM elements, and putting it into an external object, is the best method : the web page designer can then read these lists, and programmatically insert all required tables of contents/tables/figures or keyword index, at any place they wish.
For page.pdf(), this means it needs a few options more to get the page numbers into the right places:

add option numberingSelectors: accepts either one selector string, for example selectors:  'h1, h2, h3', or an array of selector strings, for example numberingSelectors:  [  'h1,h2', 'figure.indexed > figcaption', 'table>caption' ].
For each input selector, an array element will be generated into output structure
add option numberingName: name of the output structure. For example: numberingName: 'pageNumbers'
numberingPath: file name for output
alternative (or combined) to numberingPath: numberingStructure: the output structure will be written as a property with name from numberingName into this object.

A call would look like this:
  await page.pdf({
    //  ... other options, then:
    outlineDepth: 3,
    outlineExcludeSelector: 'h3.no-pdf-outline',
    numberingSelectors: [ 'h1, h2, h3' , 'figure.indexed > figcaption' ],
    numberingName: 'pageNumbers',
    numberingPath: 'elementPages.js'
  });

Output file elementPages.js would look like this:
'use strict';
var pageNumbers = [
  {
    selector: 'h1, h2, h3',
    elements: [
      { page: 2, tag: 'h1', id: 'chapter-1', innerHTML: '1. Introduction' },
      { page: 4, tag: 'h2', id: 'section-1-1', innerHTML: '1.1 Prerequisites' },
      { page: 8, tag: 'h3', id: 'section-1-1-1', innerHTML: '1.1.1 Operating System' }
    ]
  },
  {
    selector: 'figure.indexed > figcaption',
    elements: [
      { page: 3, tag: 'figcaption', id: 'fig-environment', innerHTML: 'Figure 1: Environment' },
      { page: 5, tag: 'figcaption', id: 'fig-architect', innerHTML: 'Figure 2: Architecture' },
      { page: 7, tag: 'figcaption', id: 'fig-circuit', innerHTML: 'Figure 3: Circuit diagram' }
    ]
  }
];

if (typeof module === 'object') { // Export to node.js, if available
  module.exports = pageNumbers ;
}

If outputObject is set, then:
  let someObj = {};
  await page.pdf({
    //  ... same options as previous call example, then:
      outputObject: docPages
    }
  });
  // Access page numbers and IDs with:
  for ( let i=0 ; i<someObj.pageNumbers[0].length; i++ ) {
    let someReference = someObj.pageNumbers[0].elements[i];
    // ...
  }

I believe this design is flexible and covers all relevant cases.
Some remarks:

the proposed option names may be sub-optimal (English is not my mother tongue), change them as you wish. They can be grouped into a single logical options object too.
most web programmers already used selectors with document.querySelectorAll() (options outlineExcludeSelector and numberingSelectors), therefore the semantics should be straightforward
Thanks to the selector property in output, the user can fetch all elements from DOM again, therefore only the page property is really needed.
I thought id  (if it exists) could be a useful addition.
For innerHTML: it can be useful if you cannot access the DOM directly, but may have drawbacks, especially if the elements have a lot of content. So you can remove it if you wish.
When outputting to a local object with outputObject, if there is a page DOM reference, then you can directly put a reference to the elements themselves, and keep only page in the structure.

+1
At the moment this is the one blocker for me to try this as an alternative to wkhtml2pdf. Table of contents with page numbers and navigation in the rendered pdf is a must for my use case.

Even if the ToC is non-configurable i.e it is generated using h1/h2/h3/h4 with a default depth of 4  (wkhtmltopdf defaults) it would be really helpful.

Is there an issue opened in upstream projects so that we could see if there is any progress on this ?
Thanks in advance! :)

@aslushnikov can you help us by identifying the correct upstream projects?
Looks like I'm not the only one desperately waiting for this feature :-)
For me it looks like this is the upstream entry point,
but I'm not sure.
Please share your thoughts with us...
Thanks
Ognian

@aslushnikov can you help us by identifying the correct upstream projects?

@Ognian please file to the chromium repository at https://crbug.com/new and post the link to the bug here for the record.

@aslushnikov Thanks, here it is https://bugs.chromium.org/p/chromium/issues/detail?id=840455

For those listening only on this thread, the chromium bug was just updated:
Here's the status of this:  SkPDF [*] now lets you add this information to the PDF.  Chrome does not yet use this new API.

[*] https://skia.googlesource.com/skia/+/master/include/docs/SkPDFDocument.h

We also need a Table of Contents feature. We are currently building a work around but to have this as a feature would be extremely valuable to our team.

We also need a Table of Contents feature. We are currently building a work around but to have this as a feature would be extremely valuable to our team.

Can you share how you built it?

@suportecaso1 I found this article on medium that shows a workaround to generate a pdf with TOC using chrome (puppeteer is, of course, fine too). To summarize, you could generate the pdf with the TOC but without page numbers, parse the PDF to find out the page numbers, and then regenerate it with the page numbers.
I really need a table of content in my document so I'll go with this solution for the moment, but I think this feature is a must have in puppeteer. I hope someone will implement it soon.

Any news on this feature? And about pdf internal links? because it would be a good start to generates toc without page number if the link is clickable and forward us to the actual part.

Until we wait for a native solution, I've built this simple service https://github.com/isneezy/pdf-generator-service, it can generate TOC and the solution is based on this helpful article.
Hope you enjoy it.

Hi  @simonc.  Thanks for posting the link to paged.js.  I've spent several hours trying to get paged.js to work without success.
(Also the link now goes to a 404).
Are you referring here to the TOC embedded in the PDF (also known as a "document outline dictionary")?
I wonder if you can upload an example on github, or offer a few hints?

Hey @universemaster, I can give you a watered down example of what we do in one of our master templates at PDFMonkey:

Table of Content

  {% for section in sections %}
    {% assign sectionId = forloop.index %}

{{section.title}}

  {% endfor %}

{{section.title}}

[data-page-ref]::after {
  content: 'p. ' target-counter(attr(data-page-ref), page);
}

That's pretty much it actually 🙂
Hope this can help.

I found a way to do this in just a few lines of pure HTML and JS.
This snippet is not completely reliable and will likely break on edge cases, but it worked better than paged.js for my document.
First of all, my report is split into pages which look like this:

Page Header

Then, the following JS snippet iterates over the list of all pages, counting the numbers:
window.addEventListener('beforeprint', () => {
    // style the template as needed
    // just keep .toc-link and .toc-page-number classes
    const template = '
:
';

    let result = '';
    let pageNumber = 1;
    let pageHeight = 842; // A4 size at 72 dpi

    const pages = document.querySelectorAll('.page');
    for (let p of pages) {
        const currHeight = p.getBoundingClientRect().height;

        const header = p.querySelector('.toc-header');
        if (header) {
            const tempElem = document.createElement('div');
            tempElem.innerHTML = template;

            const pageElem = tempElem.querySelector('.toc-page-number');
            pageElem.innerText = pageNumber;

            const linkElem = tempElem.querySelector('.toc-link');
            linkElem.innerText = header.innerText;
            linkElem.href = '#' + header.id;

            result += tempElem.innerHTML;
        }

        pageNumber += Math.ceil(currHeight / pageHeight);
    }

    // append result to some designated TOC container
});

Hi @impworks, Thanks for this alternative solution! I think it can work great if you're able to split your content in pages 😊
Sadly for me it's not always the case and Paged.js remains the backup plan for those situations 😅

@simonc well, it does handle sections spanning multiple pages via Math.ceil(). The page wrapper is actually needed to account for empty space before an explicit page break. Depending on the document structure, there can be some slightly different strategy.
I still wish there was a canonical built-in way to do this though, like the page number classes in a footer.

@impworks just minor bug in your code (missing "): class="toc-page-number should be class="toc-page-number"

template

Can you share a working fiddle? Or repo @impworks

@SidAngular1996 unfortunately my example above did not work as expected. It produced nice results at first, but as we modified the contents of the page the calculation became wildly unpredictable, so I do not recommend doing it this way. I was not able to come up with anything more reliable and made a decision to avoid using page numbers altogether.

@impworks hey thank you for replying.
Any idea on how can I generate toc dynamically I'm using handlebars and node js.
As my pdf gets generated based on filled up forms it's highly dynamic.
Can you suggest any package that can help me ?
Thanks in advance

Here's what I'm using (this is with pagedJS as well, running afterwards):
    function createToc(config) {
        const content = config.content;
        const tocElement = config.tocElement;
        const titleElements = config.titleElements;

        let tocElementDiv = content.querySelector(tocElement);

        // No location at which to add a Table of Contents.
        if (!tocElementDiv) {
            return;
        }
        let tocUl = document.createElement("ul");
        tocUl.id = "list-toc-generated";
        tocElementDiv.appendChild(tocUl);

        // Add class to all title elements.
        let tocElementNbr = 0;
        for (var i = 0; i < titleElements.length; i++) {

            let titleHierarchy = i + 1;
            let titleElement = content.querySelectorAll(titleElements[i]);

            titleElement.forEach(function (element) {

                // Add classes to the element.
                element.classList.add("title-element");
                element.setAttribute("data-title-level", titleHierarchy);

                // add id if doesn't exist
                tocElementNbr++;
                idElement = element.id;
                if (idElement == '') {
                    element.id = 'title-element-' + tocElementNbr;
                }
                let newIdElement = element.id;

            });

        }

        // Create toc list.
        let tocElements = content.querySelectorAll(".title-element");

        for (var i = 0; i < tocElements.length; i++) {
            let tocElement = tocElements[i];

            let tocNewLi = document.createElement("li");

            // Add class for the hierarcy of toc.
            tocNewLi.classList.add("toc-element");
            tocNewLi.classList.add("toc-element-level-" + tocElement.dataset.titleLevel);

            // Keep class of title elements except for page-break-before and -after.
            let classTocElement = tocElement.classList;
            let removedClasses = [ 'title-element', 'page-break-before', 'page-break-after' ];
            for (var n = 0; n < classTocElement.length; n++) {
                if (removedClasses.indexOf(classTocElement[n]) == -1) {
                    tocNewLi.classList.add(classTocElement[n]);
                }
            }

            // Create the element.
            tocNewLi.innerHTML = '' + tocElement.innerHTML + '';
            tocUl.appendChild(tocNewLi);
        }
    }

    window.addEventListener('DOMContentLoaded', () => {
        class handlers extends Paged.Handler {
            constructor(chunker, polisher, caller) {
                super(chunker, polisher, caller);
            }

            beforeParsed(content) {
                createToc({
                    content: content,
                    tocElement: '#toc',
                    titleElements: ['.toc_entry'],
                });
            }
        }
...

@NigelCunningham can you please share a working repo where I can build a pdf from html or extract data from a pdf I'm using handlebars and nodejs?

Here's what I'm using (this is with pagedJS as well):
    function createToc(config) {
        const content = config.content;
        const tocElement = config.tocElement;
        const titleElements = config.titleElements;

        let tocElementDiv = content.querySelector(tocElement);

        // No location at which to add a Table of Contents.
        if (!tocElementDiv) {
            return;
        }
        let tocUl = document.createElement("ul");
        tocUl.id = "list-toc-generated";
        tocElementDiv.appendChild(tocUl);

        // Add class to all title elements.
        let tocElementNbr = 0;
        for (var i = 0; i < titleElements.length; i++) {

            let titleHierarchy = i + 1;
            let titleElement = content.querySelectorAll(titleElements[i]);

            titleElement.forEach(function (element) {

                // Add classes to the element.
                element.classList.add("title-element");
                element.setAttribute("data-title-level", titleHierarchy);

                // add id if doesn't exist
                tocElementNbr++;
                idElement = element.id;
                if (idElement == '') {
                    element.id = 'title-element-' + tocElementNbr;
                }
                let newIdElement = element.id;

            });

        }

        // Create toc list.
        let tocElements = content.querySelectorAll(".title-element");

        for (var i = 0; i < tocElements.length; i++) {
            let tocElement = tocElements[i];

            let tocNewLi = document.createElement("li");

            // Add class for the hierarcy of toc.
            tocNewLi.classList.add("toc-element");
            tocNewLi.classList.add("toc-element-level-" + tocElement.dataset.titleLevel);

            // Keep class of title elements except for page-break-before and -after.
            let classTocElement = tocElement.classList;
            let removedClasses = [ 'title-element', 'page-break-before', 'page-break-after' ];
            for (var n = 0; n < classTocElement.length; n++) {
                if (removedClasses.indexOf(classTocElement[n]) == -1) {
                    tocNewLi.classList.add(classTocElement[n]);
                }
            }

            // Create the element.
            tocNewLi.innerHTML = '' + tocElement.innerHTML + '';
            tocUl.appendChild(tocNewLi);
        }
    }

    window.addEventListener('DOMContentLoaded', () => {
        class handlers extends Paged.Handler {
            constructor(chunker, polisher, caller) {
                super(chunker, polisher, caller);
            }

            beforeParsed(content) {
                createToc({
                    content: content,
                    tocElement: '#toc',
                    titleElements: ['.toc_entry'],
                });
            }
        }
...

I know it has been a while but I was wondering if you would know why this approach doesn't work when trying to use it with bootstrap.
In my example html, I have latest bootstrap version with paged polyfill and for some reason it doesn't want to generate the PDF, it is always blank, if I remove either, it works.


Add pyramid

All commits 273-275 will have merge conflicts. Please fix them.
These were caused by you adding blank lines in the table in the Java.md file that I fixed earlier.

@ajpmaclean : Conflicts Resolved. Please review.

Thanks