Added count-args to simple-pipe

added a custom action in separate file called count-args

fixed the path issue!


Update to dreq 01.00.24

@dnadeau4 @doutriaux1 we're behind the latest dreq version 01.00.24, with 01.00.23 the current version of these tables. It would be great to update these to the latest.

@durack1 I have done this in my half level branch, plan on pushing this today.

@durack1 actually there's some noise about these see and chime at: https://github.com/PCMDI/cmor/issues/355

Before the update to data request version 01.00.24 is released could you ensure that the long name of the longitude in the CMIP6_coordinate.json file (line 873 in #182) is set to longitude. The error in the data request was identified in issue cmip6dr/Request#82
I think this issue could result in an incorrect long name in the vast majority of output files produced using version 01.00.24 if not fixed.

@taylor13 ping

@matthew-mizielinski @doutriaux1 Looks like long name is now "Longitude", so I think this can be closed.


Intellisense is not working in react native project

Issue Type: Bug
intellisense is not work in react native
Extension version: 1.3.0
VS Code version: Code 1.52.1 (ea3859d4ba2f3e577a159bc91e3074c5d85c0523, 2020-12-16T16:34:46.910Z)
OS version: Windows_NT x64 10.0.19042

System Info

Item
Value

CPUs
AMD PRO A4-4350B R4, 5 COMPUTE CORES 2C+3G      (2 x 2495)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: disabled_offoop_rasterization: disabled_offopengl: enabled_onprotected_video_decode: unavailable_offrasterization: enabledskia_renderer: disabled_off_okvideo_decode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
3.47GB (0.98GB free)

Process Argv
--crash-reporter-id d21752c6-772b-4340-a089-06376a4c9802

Screen Reader
no

VM
0%

A/B Experiments
vsliv368cf:30146710
vsreu685:30147344
openlogontheside:30221877
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
vswsl492cf:30211402
wsl2promptcf:30224613
pythonvsdeb440:30248342
unusedpromptcf:30224611
folderexplorercf:30224615
openfilemenucf:30224648
pythonvsded773:30248341
pythonvspyt600:30247031

![Screenshot

(13)](https://user-images.githubusercontent.com/64305855/105825826-0707ca80-5fe6-11eb-8b16-2120d220c712.png)

(Experimental duplicate detection)
Thanks for submitting this issue. Please also check if it is already covered by an existing one, like:

Disable intellisense when typing <?php in php documents (#68535)
intellisense not working (#109208)

I told is not working in react native project
Please see above screen shots

(Experimental duplicate detection)
Thanks for submitting this issue. Please also check if it is already covered by an existing one, like:

Disable intellisense when typing <?php in php documents (#68535)
intellisense not working (#109208)

I told is not working in react native project
Please see above screen shots

How to fix it .
My problem is if I type padd means it want to give options to me like paddingLeft, paddingRight,paddingHorizontal,paddingVertical .
But it is not working why how to fix it
You can see it in my images above

How to fix it .
My problem is if I type padd means it want to give options to me like paddingLeft, paddingRight,paddingHorizontal,paddingVertical .
But it is not working why how to fix it
You can see it in my images above

Seems to work for me:

Please share a small example project that demonstrates the issues
/needsMoreInfo

Seems to work for me:

Please share a small example project that demonstrates the issues
/needsMoreInfo

Its working fine in my linux
But I need it in windows
I noticed one thing while using in linux
I create the react then I was open the folder using vs code
After that it will automatically download one zip file name is intellicode xxxxxx.zip something I don't know exactly
but in my windows I  uninstalled vs code and then install it again it was show same I feel like notepad without intellisense

So can you tell me how to full  uninstall vs code
I will install it again then try it

Now its working sir thank you
I was reset my pc then I install again node and npm ,vs code now it working

Thank you very much sir

Seems to work for me:

Please share a small example project that demonstrates the issues
/needsMoreInfo

Can you suggest me usefull extensions for react native


Create develop branches

This was only done for Kitura main, we need to do it for the other repos as well

So I was starting to look over this development branch. Not sure if this warrants opening another issue or just following on this issue.
Both on master and develop, Package.swift is the same and contains:
dependencies: [
        .Package(url: "https://github.com/IBM-Swift/Kitura-router.git", versions: Version(0,2,0)..<Version(0,3,0)),
        //.Package(url: "git@github.com:IBM-Swift/Kitura-router.git", majorVersion: 0),
        .Package(url: "https://github.com/IBM-Swift/HeliumLogger.git", versions: Version(0,2,0)..<Version(0,3,0)),
        //.Package(url: "git@github.com:IBM-Swift/HeliumLogger.git", versions: Version(0,0,0)..<Version(0,1,0)),
        .Package(url: "https://github.com/IBM-Swift/LoggerAPI.git", versions: Version(0,2,0)..<Version(0,3,0)),
        //.Package(url: "git@github.com:IBM-Swift/LoggerAPI.git", majorVersion: 0),
        .Package(url: "https://github.com/IBM-Swift/Kitura-TestFramework.git", versions: Version(0,2,0)..<Version(0,3,0)),
        //.Package(url: "git@github.com:IBM-Swift/Kitura-TestFramework.git", majorVersion: 0),
        ]

We need to keep master the way it is- but develop should be changed to:
dependencies: [
        .Package(url: "https://github.com/IBM-Swift/Kitura-router.git", versions: Version(0,3,0)..<Version(0,4,0)),
        //.Package(url: "git@github.com:IBM-Swift/Kitura-router.git", majorVersion: 0),
        .Package(url: "https://github.com/IBM-Swift/HeliumLogger.git", versions: Version(0,3,0)..<Version(0,4,0)),
        //.Package(url: "git@github.com:IBM-Swift/HeliumLogger.git", versions: Version(0,0,0)..<Version(0,1,0)),
        .Package(url: "https://github.com/IBM-Swift/LoggerAPI.git", versions: Version(0,2,0)..<Version(0,3,0)),
        //.Package(url: "git@github.com:IBM-Swift/LoggerAPI.git", majorVersion: 0),
        .Package(url: "https://github.com/IBM-Swift/Kitura-TestFramework.git", versions: Version(0,2,0)..<Version(0,3,0)),
        //.Package(url: "git@github.com:IBM-Swift/Kitura-TestFramework.git", majorVersion: 0),
        ]

This will allow our 0.3.x development tags to be picked up on the develop branch. Similar changes need to be made to Kitura-router, Kitura-net, and Kitura-sys


INTEXT-180 Fetch pool size should be based on the number of brokers

Merging as is after some internal discussion

Merged as https://github.com/spring-projects/spring-integration-kafka/commit/d91e2efe704015d0e04e8981808ad7379dd6b31f after rebase


Improve usability of 'scheduled arrival'

One idea:

If no real-time info, in the prominent spot that displays how long until the bus comes, just put a "?".  Then underneath, where it currently says for example "2:31pm - scheduled arrival*" instead say "scheduled arrival in 5m at 2:31pm*"  Still sort these by the scheduled arrival time, and still have the * footnote about what this means.

Also, worth considering looking at the experience provided by other similar apps, which adorn real time arrivals/departures with a little 'communicating' icon.

#1055 This might be related to this issue.


Support HTTP/2 proxy backends

Look at nghttpx for inspiration.
If Caddy supported HTTP/2 backends for reverse proxying you could use it to route GRPC connections to the relevant servers. GRPC has the opposite (route non-GRPC requests to a webserver) using cmux I think, might be worth looking into.

With Go 1.6, this should be the case, since http2 is enabled by default for clients. The next release of Caddy, slated to come out this week, will be built on Go 1.6.


TrueNAS SCALE support?

Not sure if this is the right place to do so but it'd be great if I can install this one on my TrueNAS SCALE NAS server - Tried pulling the image directly from its Docker Compose tool but it didn't manage to deploy.
TrueNAS SCALE supports 3rd-party apps via its docker/k3s environment and there are already a good 3rd-party repository called TrueCharts, may be you could give them a call and make this happen.

Interesting, did TrueNAS give you any logs or indication of why the Docker-compose stack didn't deploy properly?
I'll have to look at the TrueCharts, looks like they are essentially helm charts? I'm currently looking at installing TrueNAS on my QNAP, so I could look at it then

First of all, please forgive my poor English. I will continue to work hard to learn English
I get the same problem，I try to run this yaml with “docker-compose” in TrueNAS.But it is not worked.Is there any suggestion？
Looking forward to receiving a reply.Thanks！
==========================================Detail=================================
TrueNAS Scale Version: 22.12.2
TrueCharts docker-compose version: 23.0.6_6.0.14
k8s describe command output:
<EMAIL_ADDRESS>k3s kubectl describe pod davinci-docker-compose-7466ffbb85-5xqdc -n ix-davinci
Name:             davinci-docker-compose-7466ffbb85-5xqdc
Namespace:        ix-davinci
Priority:         0
Service Account:  default
Node:
Labels:           app=docker-compose-6.0.14
app.kubernetes.io/instance=davinci
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=docker-compose
app.kubernetes.io/version=23.0.6
helm-revision=3
helm.sh/chart=docker-compose-6.0.14
pod-template-hash=7466ffbb85
pod.name=main
release=davinci
Annotations:      rollme: TjGUk
Status:           Pending
IP:
IPs:
Controlled By:    ReplicaSet/davinci-docker-compose-7466ffbb85
Containers:
davinci-docker-compose:
Image:      tccr.io/truecharts/docker-in-docker:v23.0.6@sha256:f0392a7e7a7b9f977733bea6904754332e8e8279bad0be3437b6ae845cbf1a40
Port:       2376/TCP
Host Port:  0/TCP
Limits:
cpu:     4
memory:  8Gi
Requests:
cpu:      10m
memory:   50Mi
Liveness:   tcp-socket :2376 delay=10s timeout=5s period=10s #success=1 #failure=5
Readiness:  tcp-socket :2376 delay=10s timeout=5s period=10s #success=2 #failure=5
Startup:    tcp-socket :2376 delay=10s timeout=2s period=5s #success=1 #failure=60
Environment:
TZ:                      Asia/Shanghai
UMASK:                   0022
UMASK_SET:               0022
NVIDIA_VISIBLE_DEVICES:  void
PUID:                    568
USER_ID:                 568
UID:                     568
PGID:                    0
GROUP_ID:                0
GID:                     0
COMPOSE_FILE:            /mnt/NoRAID_18T/applications/Docker/Davinci/docker-compose.yml
DOCKER_TLS_CERTDIR:      /certs
Mounts:
/config from docker-certs-ca (rw)
/dev/shm from devshm (rw)
/docker-certs-ca from persist-list-0 (rw)
/shared from shared (rw)
/tmp from tmp (rw)
/var/logs from varlogs (rw)
/var/run from varrun (rw)
Conditions:
Type           Status
PodScheduled   False
Volumes:
devshm:
Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:     Memory
SizeLimit:
docker-certs-ca:
Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName:  davinci-docker-compose-docker-certs-ca
ReadOnly:   false
persist-list-0:
Type:          HostPath (bare host directory volume)
Path:          /mnt/NoRAID_18T/applications/Docker/Davinci/cert
HostPathType:
shared:
Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit:
tmp:
Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit:
varlogs:
Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit:
varrun:
Type:        EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:      Memory
SizeLimit:
QoS Class:       Burstable
Node-Selectors:
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type     Reason            Age   From               Message

Warning  FailedScheduling  2m1s  default-scheduler  0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling.

Just an update, I finally got around to installing TrueNAS on my QNAP, so I'm starting to look at this again!

I am also a TrueNAS Scale user and interested in running this as a Docker Image.  Any help I can offer please let me know and I will be happy to assist as best I can.

I'm hoping to take some time this weekend to look at this! I'm still learning about TrueNAS charts and how exactly they differ from Helm charts.

Still planning on working on this, just been busy with some personal stuff!

I think this would likely be best served by another project, since this won't use docker-compose at all. I'm going to close this issue, and start working on that here

Just saw this, looks like TrueNAS is making a shift towards Docker Compose, so I'll reevaluate once Electric Eel is released! Given this, I probably won't put any work into a k3s version given this information

Still waiting for the final release of TrueNAS 24.10, then I can work on this

Now that 24.10 is out, I'll start taking a look at this in the next few weeks!


Use el-patch to install irony-server

We are sort of overriding irony--find-server-executable. Using el-patch would future-proof this.

Irrelevant since 61fd2056fba50561e3a0e719e68b6ed19d8adef2.


Add verbose mode to Reconciler

to print some logs to STDOUT.

@magnusvk thanks for your valuable feedback. I changed the code and also added a spec for it. Please have a look at it again and let me know what you think 👀

@ChrisBr thanks! Going to release this as 2.2.0 when the specs go green.


Feature Request (sort of) - MDM Workflow should log the Jamf Pro/Cloud server version

To help with debugging, if the MDM workflow is being used, super should also log the Jamf Pro/Cloud server version, the version of the jamf command and/or the Self Service app. This should make it easier to identify if the issue in an MDM workflow is a known issue with the Jamf Pro/Cloud version (and reduce the need to ask for the version in the slack channel).
The jamf command, and Self Service should always match the server version, unless the server was just updated, but even then, the on computer version should self update within a day of the server updating.
It should also be possible to gather the server version through the connection to the server when sending the MDM commands.

https://github.com/Macjutsu/super/releases/tag/v3.0


fix: display label instead of value in select column in table widget #26188

Fixed issue 26188
This pull request adds a new feature called "Display as" for the select column type in the table widget.

Whenever a column is set to select, by default we show labels and not values.

For existing tables, since we have created this problem we ensure we don't break their apps. We will give an option called "Display As" as suggested in the original issue and for new select columns, it will be Label and for existing columns, it will be Value.

We make it clear that the computed value here refers to the default value/display value. Displaying this as the helper text below the computed value property only for select column type.

Added a helper text below the computed value field that indicates that the computed value here is a default value/display value

Validation:When computed value is not present in the select option then we throw an error in select option. This is irrespective of displayAs property.

-For new row select options, there is no validation of computed value. Since are adding new data into the table and it has to adhere to the select options that we provide.

For displaying value in cell,
When display as is set to label, we show empty values for missing labels in the widget

When display as is set to value, we show the computed value as it is even if labels are missing

When computed value is not present in the select options, we still make the select options available. We do this because its not the select options that has issue but its computed value.

Summary by CodeRabbit

New Features

Enhanced SelectCell component to allow customization of display behavior, showing either the label or value of selected options.

Bug Fixes

Fixed issues related to the display properties of select columns in the TableWidgetV2 component.

Tests

Added tests for the display properties of select columns in the TableWidgetV2 component.

Refactor

Updated property configurations and utility functions to improve handling of select column display properties.

Hey sorry but I am closing the PR as TableWidget is a very complex widget and requires a lot of context to fix bugs in it.
It would be great if you could pick issues marked with a "Good first" label.


update ion url

change to our new host

@nitro-neal got all of them there is a test in the sdk which I'll cover separately


Try checking your system time http://stackoverflow.com/questions/20081916/github-shows-wrong-time-of-commits

My hackintosh have a wrong system time, but I merged from the browser, anyway I think is not a huge problem


Inquiry Regarding Dataset Release Date

Hope this message finds you well and thank you for your great work. We are very interested in your research and the accompanying dataset that you've mentioned in your recent publication. We believe it could significantly enhance our ongoing work.
Could you kindly share the anticipated timeline for the dataset release? Any information you can provide would be greatly appreciated.

Hi,
Thanks for expressing your interest in our work. Please note, that the dataset will be made publically available by May 1st, 2024.


Soft-suggesting markdown for formatting

The textual fields will inevitably need some formatting (lists, code samples etc.).
Rather than forcing any specific markup language, the Application type should suggest using one that's possible to render
in plain text, with a nod towards markdown as example of this.

What about soft-requiring it to be Markdown? The issue I see with suggesting Markdown as an example of plain-text friendly markup, is that someone could instead choose to use reStructuredText, and now a UI has the job of figuring out which one is being used. Plain-text is valid Markdown, so just describing that the field should be Markdown is enough IMO (but describing the field should be "either plain-text or Markdown" is also OK).
I would argue that Description should specifically not be formatted text as it is supposed to be brief. For comparison, in charts we only expect the README to be Markdown, description and notes are expected to only be plain-text. I can see the value in Notes being formatted, however.

I agree with this suggestion. I'd rather the tools didn't have to sniff for the markup language. I'm glad to see the idea to bless a markup is welcome.

/approve

@konryd did you want to update this PR to address the above?

The OLM manifests supported Markdown in the description for a while and we used them to render a large portion of the page on the Console. It was nice, but not all tools in the pipeline are going to support rendering markdown. Importantly, if you want to front your applications with an Open Service Broker, that spec isn't going to be able to leverage your description if it's formatted in markdown.

Update: this branch was created from github's website, which resulted in a fork that lives in the repo. Moved to my own fork now, but it requires a separate PR. It is here: https://github.com/kubernetes-sigs/application/pull/53


feature: https 支持

Codecov Report

Merging #143 (af55cfc) into v2.6.x_develop (2a72654) will not change coverage.
The diff coverage is 0.00%.

@@               Coverage Diff               @@
##           v2.6.x_develop     #143   +/-   ##
===============================================
  Coverage           51.00%   51.00%
===============================================
  Files                 693      693
  Lines               36709    36709
===============================================
  Hits                18724    18724
  Misses              17985    17985

Impacted Files
Coverage Δ

itsm/sites/views.py
48.64% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2a72654...af55cfc. Read the comment docs.


Upgrade to PureScript 0.15

Hi @athanclark do you want an upgrade?

@athanclark will you be accepting this pull request? There are additions like HKDF & PBKDF2 support and updates to tests due to recent browser versions that require updates as well. Will send you additional PRs but they depend on this one.

@sumo please make a PR to this repo when you're ready. Even if your PR never gets merged, then other people will see your branch and be able to use your work.

I forked this repo to https://github.com/rowtype-yoga/purescript-yoga-subtlecrypto


ibm watson do not show that device is connected

Hello, I am trying to make IoT app based on esp8266, node-red and IBM cloud. I follow your guide.  I only use different sensors.
I had to solve some problems. I had to change the order of configTime() and Wifi,begin(). In your order, it didn't work me.
Now I am at part 2: IoT Security. I configured the IBM cloud and uploaded the certificate to esp. The ESP prints "MQTT connected" and begins to send data, but IBM Watson says that the device is not connected, but in section, connection protocols appear new message (attachment). It worked without TLS. I do not know what I did wrong. I append serial output and my code.
I'm not sure if I added the certificate to chrome correctly.
Pavel
Connected to Miroslav
IP address: <IP_ADDRESS>
Setup done
Loaded CA cert
Loaded key
Loaded client cert

Waiting for time
Time sync'd
Sun Oct 18 00:57:38 2020

MQTT Connected
{"d":{"temp":24.14,"humidity":61.1,"water_level":18.326,"soil_humidity":0}}
{"d":{"temp":24.14,"humidity":60.95,"water_level":18.343,"soil_humidity":0}}
{"d":{"temp":24.15,"humidity":61.02,"water_level":17.918,"soil_humidity":0}}
...

data_aquisition.ino.txt

I found that the mqtt.status() returns 1 (MQTT_CONNECT_BAD_PROTOCOL). I am not sure what I have to do.

I found where is the problem, but I don't know how to solve it. Chrome do no use my CA certificate. I tried to install firefox, and everything works like. I tried to add it via MMC and directly in chrome.

Depending in the OS you are using you may need to add your certificate to your OS root certificates, or to the browser.
In Chrome browser settings there is a Privacy and Security section, where you should find the option to Manage Certificates.  You need to ensure your root CA certificate is added to the trusted root certificates.


Truncate post slugs when importing from Tumblr.

Fixes #481.

Going to close this and open two separate requests.


pull request for e2e test at 12/10/2016 07:15:02

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

optest/docs/updated_at.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Add docs for tags in notebooks

Add documentation update after #51

Ok - after looking at https://github.com/tradingstrategy-ai/docs/blob/master/source/conf.py I realized the only difference was that they use nbsphinx and I was using myst_nb. Indeed, it seems like this is a known problem: MyST-nb doesn't support the display of raw cells.
I will add a note in the docs about this, merge these docs and cut a release.


Tooltip inside a Button is not working only in Firefox

<button type={"button"} >
<i data-tip={"tooltip here"} data-for="widget-tooltip" className={"fa fa-info"}>

This is a browser level bug and a dup of https://github.com/wwayne/react-tooltip/issues/296


I think there is a problem with StructScan design

When I use this method, I want to convert a DB row to Struct Object.
I found a mistake in it.
I would like to know why you are considering using missingFields.
If the number of DB row columns exceeds that of struct columns,
Why do you have to make missing field mandatory?
As a business client I really don't understand.

I think that's exactly the right design. If there's nowhere for the data to go, change your query to only return fields that you care about


opencv4nodejs install error on macos intel - binding.gyp not found

opencv4nodejs install error on macos intel.
Is there something missing?
Error Message
><EMAIL_ADDRESS>install /opencv-test/node_modules/@u4/opencv4nodejs
> node-gyp rebuild

gyp: binding.gyp not found (cwd: /opencv-test/node_modules/@u4/opencv4nodejs) while trying to load binding.gyp

OpenCV version : 4.5.4_4
With OpenCV-contrib? (extra modules): no
OS: MacOSX Intel

I am also having this issue.

Seems that node-gyp is trying to rebuild on install. I had this working around exactly 20 days ago so I am guessing that it may be to do with the last commit. My method to get it working on an intel Mac was quite convoluted at that time:
npm install --save @u4/opencv4nodejs
Add the script lines to the package.json
npm install -g node-gyp
inside the @u4/opencv4nodejs directory in node_modules rename _binding.gyp to binding.gyp
Go to root folder
npm run do-install
However, I can't get past the initial npm install now, and none of the switches seem to disable the node-gyp rebuild command (except --dry-run, which is just a series of logs to the console).

binding.gyp is hidden by default as _binding.gyp the build script copy it as binding.gyp during the custom build process.

@UrielCh hi, could u install the package successfully on Nodejs v12.20.1?
and what does the custom build mean?
I just install the package with npm install @u4/opencv4nodejs, but it does not work.

See opencv4nodejs/install/compileLib.ts
This script properly configure environment before calling node-gyp rebuild.
see:
  "bin": {
    "build-opencv": "bin/install.js"
  },

in package.json

Version 6.1.2 is ready

@UrielCh Tanks for the replay and work.
I try to install the<EMAIL_ADDRESS>but install script is broken.
> echo Use 'build-opencv rebuild' of (node bin/install.js rebuild) script to start node-gyp, use --help to check all options.

sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `echo Use 'build-opencv rebuild' of (node bin/install.js rebuild) script to start node-gyp, use --help to check all options.'

Maybe the () need \, like
echo Use 'build-opencv rebuild' of \(node bin/install.js rebuild\) script to start node-gyp, use --help to check all options.

I send a PR.
Feel free to edit.

noooo, I need to made a new release :(

Is the original issue still exists ?
My Intel based MacOS is in a cupboard...

I will try again tonight, sorry for the delay replay.

@UrielCh The original issue has been fixed, the installation can be completed normally, thanks.
The next problem is that the compilation of opencv fails.
It seems that this library cannot use the opencv installed by brew. I will open a new issue after trying it.


Support for dictionaries as observations?

Does LightZero provide support for complex observation types, such as dictionaries of multiple tensors? If not, where should the change be made? Right now I'm thinking:

hope that all tensors in the dictionary are of the same type
join them and flatten the result
let LZero do its thing
unflatten the flattened tensor merge inside a custom model implementation (override model class)

In this PR39, we have put the emphasis on managing complex structured observations. The current main branch does not yet provide support for dict-type observations. However, PR39 provides an good reference to devise a specialized encoder for dictionary-type observations. Once developed, this encoder can replace the main backbone of the representation network (as indicated in common.py). Moreover, you might need to make minor adjustments in the data preprocessing code specifically in game_buffer_muzero.py. For guidance, you can refer to the modifications made in this PR39. By incorporating these changes, you can effectively manage more complex structured observation data.
Please note that the modifications in PR39 are intended for handling more intricate structured observations and are for reference only.


Generate nx-stylelint for a application give error Expected argument jsonString to be a string, got object

Hello,
Situation:
NX workspace is updated from 9.8.0 to 12.x. Added eslint and now added nx-stylelint.
Now trying to add styelint to a project with command:
ng generate nx-stylelint:configuration --project=my-app --skipFormat --no-interactive --dry-run
which resuilt in this error: Expected argument jsonString to be a string, got object
Any idea's how to solve that?
Cheers,
Ringo.

Hi Ringo,
Please run nx report and provide the output. I will try to reproduce the issue when i get your version details.
Thanks

Please also provide the full stacktrace of the error.

Hi @Phillip9587 ,
I have the same problem as @ringodotnl with Nx 12
Report:
  Node : 14.16.1
  OS   : darwin x64
  yarn : 1.22.10

  nx : 12.0.6
  @nrwl/angular : Not Found
  @nrwl/cli : 12.3.3
  @nrwl/cypress : 12.3.3
  @nrwl/devkit : 12.3.3
  @nrwl/eslint-plugin-nx : 12.3.3
  @nrwl/express : Not Found
  @nrwl/jest : 12.3.3
  @nrwl/linter : 12.3.3
  @nrwl/nest : Not Found
  @nrwl/next : 12.3.3
  @nrwl/node : 12.3.4
  @nrwl/react : 12.3.3
  @nrwl/schematics : Not Found
  @nrwl/tao : 12.3.3
  @nrwl/web : 12.3.3
  @nrwl/workspace : 12.3.3
  @nrwl/storybook : 12.3.3
  @nrwl/gatsby : Not Found
  typescript : 4.2.4

how can I provide you a full stacktrace of the error?
Thanks!

@ild0tt0re Please try updating your global Nx installation with npm i -g nx@latest.

@ild0tt0re Please also provide your stylelint version.

@ild0tt0re Please also provide your stylelint version.

All my stylelint stuff:
├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>└─<EMAIL_ADDRESS>

I had the same issues with nx-stylelint@12.
nx-stylelint@11 seems to work for me

@ringodotnl @ild0tt0re @cogoo Can someone please copy the full error message and details.
I cannot reproduce the issue. I tested it on linux and windows but i have no mac to test on. I used the latest create-nx-workspace to generate a new Nx workspace and added nx-stylelint. The init and configuration generators worked.

@Phillip9587 this was the full extent of the error message:
Expected argument jsonString to be a string, got object

I'll do some debugging later tonight and share my findings.
P.S Thanks for this Nx plugin, its awesome!

I added e2e tests and a github workflow which runs the e2e test on linux, windows and mac on with node version 12, 14 and 16. I still don't get any errors while running the generators.
I just released version 12.1.0 of nx-stylelint maybe you can try it out?

Same error:

Nx report:
  Node : 12.22.1
  OS   : darwin x64
  npm  : 6.14.12

  nx : Not Found
  @nrwl/angular : 11.6.3
  @nrwl/cli : 11.6.3
  @nrwl/cypress : 11.6.3
  @nrwl/devkit : 11.6.3
  @nrwl/eslint-plugin-nx : 11.6.3
  @nrwl/express : Not Found
  @nrwl/jest : 11.6.3
  @nrwl/linter : 11.6.3
  @nrwl/nest : Not Found
  @nrwl/next : Not Found
  @nrwl/node : Not Found
  @nrwl/react : Not Found
  @nrwl/schematics : Not Found
  @nrwl/tao : 11.6.3
  @nrwl/web : Not Found
  @nrwl/workspace : 11.6.3
  @nrwl/storybook : 11.6.3
  @nrwl/gatsby : Not Found
  typescript : 4.0.7

@cogoo Please use nx-stylelint 11 or update to Nx 12. nx-stylelint's major version has to match the major version of Nx. Thanks

@cogoo Please use nx-stylelint 11 or update to Nx 12. nx-stylelint's major version has to match the major version of Nx. Thanks

awesome, thanks thats what i observed. nx-stylelint 11 works on my repo

@ild0tt0re Do you still have problems with nx-stylelint?

Fixed! Thanks!


Optionally pass config name to RodauthMailer methods so @email_link includes the config prefix

I created an admin account configuration like this:
class RodauthAdmin < RodauthBase # inherit common settings
  configure do
    # List of authentication features that are loaded.
    prefix "/admin"
  end

And the verification email link that was generated upon account creation didn't include /admin in the path.
Currently, there doesn't seem to be a way to pass in configuration names (like :admin) to methods in RodauthMailer that can be used to call #RodauthApp.rodauth so the correct path for the configuration is generated.
I quickly made the following changes to the app/mailers/rodauth_mailer.rb. There might be a better way to do this.
I'm posting this as a draft for your consideration. I'm happy to help contribute to this project in any way I can.

Thanks for the pull request. That's a valid point, I agree it would be useful to be able to just move mailer hooks into the base configuration, and have it automatically work with multiple configurations.
I was wondering whether it wouldn't be better as a first argument, given that it's something passed into every mailer method, though I don't know if Ruby supports the first argument being optional 🤔

BTW, you can access the configuration name via self.class.configuration_name inside the auth method blocks.


elixir-ls language server does not start

Information
VIM version
NVIM v0.3.1

OS
Arch Linux
What went wrong

elixir-ls doesn't start when editing an elixir file, and manually running :ALELint does nothing. My g:ale_elixir_elixir_ls_release variable is set correctly. This may have to do with me using asdf to manage elixir versions, although manually running the language server executable from vim with !~/.language-servers/elixir-ls-release/language_server.sh does work.
:ALEInfo
 Current Filetype: elixir
Available Linters: ['credo', 'dialyxir', 'dogma', 'elixir-ls', 'mix']
  Enabled Linters: ['elixir-ls']
 Suggested Fixers:
  'mix_format' - Apply mix format to a file.
  'remove_trailing_lines' - Remove all blank lines at the end of a file.
  'trim_whitespace' - Remove all trailing whitespace characters at the end of every line.
 Linter Variables:

 Global Variables:

let g:ale_cache_executable_check_failures = v:null
let g:ale_change_sign_column_color = 0
let g:ale_command_wrapper = v:null
let g:ale_completion_delay = 100
let g:ale_completion_enabled = 1
let g:ale_completion_max_suggestions = 50
let g:ale_echo_cursor = 1
let g:ale_echo_msg_error_str = 'Error'
let g:ale_echo_msg_format = '%code: %%s'
let g:ale_echo_msg_info_str = 'Info'
let g:ale_echo_msg_warning_str = 'Warning'
let g:ale_enabled = 1
let g:ale_fix_on_save = 1
let g:ale_fixers = {'reason': ['refmt'], 'elixir': ['mix_format'], 'vue': ['prettier', 'eslint'], 'html': ['prettier'], 'javascript': ['prettier', 'eslint'], 'ocaml': ['ocamlformat'], 'python': ['yapf']}
let g:ale_history_enabled = 1
let g:ale_history_log_output = 1
let g:ale_keep_list_window_open = 0
let g:ale_lint_delay = 200
let g:ale_lint_on_enter = 1
let g:ale_lint_on_filetype_changed = 1
let g:ale_lint_on_insert_leave = 0
let g:ale_lint_on_save = 1
let g:ale_lint_on_text_changed = 'always'
let g:ale_linter_aliases = {}
let g:ale_linters = {'reason': ['merlin'], 'elixir': ['elixir-ls'], 'javascript': ['eslint'], 'ocaml': ['merlin']}
let g:ale_linters_explicit = 0
let g:ale_list_vertical = 0
let g:ale_list_window_size = 10
let g:ale_loclist_msg_format = '%code: %%s'
let g:ale_lsp_root = {}
let g:ale_max_buffer_history_size = 20
let g:ale_max_signs = -1
let g:ale_maximum_file_size = v:null
let g:ale_open_list = 0
let g:ale_pattern_options = v:null
let g:ale_pattern_options_enabled = v:null
let g:ale_set_balloons = 0
let g:ale_set_highlights = 1
let g:ale_set_loclist = 1
let g:ale_set_quickfix = 0
let g:ale_set_signs = 1
let g:ale_sign_column_always = 0
let g:ale_sign_error = '>>'
let g:ale_sign_info = '--'
let g:ale_sign_offset = 1000000
let g:ale_sign_style_error = '>>'
let g:ale_sign_style_warning = '--'
let g:ale_sign_warning = '--'
let g:ale_statusline_format = v:null
let g:ale_type_map = {}
let g:ale_use_global_executables = v:null
let g:ale_virtualtext_cursor = 0
let g:ale_warn_about_trailing_blank_lines = 1
let g:ale_warn_about_trailing_whitespace = 1
  Command History:
**[[THIS IS EMPTY]]**

yeah I am in this same boat. I have done all the configuration I can think of and when I do ALEInfo it never shows it trying to talk to the ls

actually might have found the issue for me. If I set the g:ale_elixir_elixir_ls_release to a path that contains home as ~ then it cant find it. When a do the path spelled out it works fine.

I don't know if this is the right way to handle this (new to vim) but having it this way in my vimrc worked
let g:ale_elixir_elixir_ls_release = $HOME . '/elixir-ls/rel'

I didn't want to hard code the home patch because on my machines its a little different. If I have something wrong here I would like the feedback. Or if this is something that is a known issue we should get it documented

After I replaced the ~ with a hard-coded path ALE does start the elixir-ls server, but I'm still not getting any errors.

and :ALEGoToDefinition isn't working?

It appears to be working part of the time...

I got mine working consistently, it takes a sec to start up but it works afterwards. If you want to bounce ideas off of each other to see if its something in your vimrc let me know.

My experience: the first time I run :ALEGoToDefinition, the CPU goes to 50% for 45 seconds or so, and nothing happens in the vim UI.  After that, :ALEGoToDefinition works as expected, sub-second response time.  (Note: I had to use an absolute path for the elixir-ls/rel directory...)

I did notice a slow start up. Tomorrow I will run again with diagnostics and see if I get the same cpu spike.

Seeing the same thing here. Anyone closer to figuring this out?

Make sure that the elixir-ls was built with the same elixir and erlang versions that your project is using.

I got this to work with the following...and it is awesome.

What @mhanberg said is spot on
asdf install elixir ref:v1.81 # I think the important part having the source for ale and elixir_ls to look up.
asdf global elixir ref:v1.8.1
let g:ale_elixir_elixir_ls_release = $HOME.'/lsps/elixir-ls/rel'. # obviously you're path would be diff.

For listing make sure you have your doing fixers and linters.

You might also want to try setting b:ale_lsp_root to some project root path, if ALE can't discover the project root automatically.

@mhanberg's solution works for me. Can we close?

Yes, that makes sense. You can also use b:ale_elixir_elixir_ls_release in an ftplugin file so you can set different releases for different projects.


My Multi-Clip Reward implementation results.

Hi,
I have tried to implement the Multi-Clip reward technique, but so far I have not achieved great results. My goal is to reproduce the steering behavior shown in the document.
I made the following steps:
1] Added loading of multiple MoCap clips (walk_forward, turn_left0, turn_left1, turn_left2, turn_right0, turn_right1, turn_right2). For convenience, I've modified them in order to have the same number of frames and the same frame rate (30 fps) and so the same duration. I think this is important, because we have a single controller and I don't know which of the 7 clips to choose (for example) to set the controller cycle period (see SceneImitate::BuildController method).
All kinematic actors are perfectly synchronized as you can see from this video:
Https://youtu.be/MXAnrLQL2fo
As usual, when a clip reachs its end, the kinematic actor is repositioned and reoriented according to the position and orientation of the simulated actor (in green, not shown in this video).
One question: all motion clips have the "wrap" flag set to "loop". Is it correct? Or should I set to "loop" only the "walk forward"? Can this affect the result?
2] Added the heading goal to the network through 2 floats representing a unit vector (dx, 0, dz): it only represents the direction.
3] Setted the sync_char_root_rot flag to true (in addition to the sync_char_root_pos). I think this is necessary, otherwise when the actor will be aligned with the direction vector, it will not select the walk forward. Am I right?
4] Added two functions to the SceneImitate.cpp:
double cSceneImitate::CalcKinHeadingRewardGoal(const cSimCharacter& sim_char, const cKinCharacter& kin_char) const
{
	double reward = 0.0;
	auto ct_ctrl = dynamic_cast<cCtController *>(this->GetController().get());
	if (ct_ctrl != nullptr)
	{
		const auto goalDir = ct_ctrl->GetGoalDir();

		const auto& joint_mat = sim_char.GetJointMat();
		const auto& body_defs = sim_char.GetBodyDefs();

		const Eigen::VectorXd& pose1 = kin_char.GetPose();
		const Eigen::VectorXd& vel1 = kin_char.GetVel();

		tVector com1_world;
		tVector com_vel1_world;
		cRBDUtil::CalcCoM(joint_mat, body_defs, pose1, vel1, com1_world, com_vel1_world);

		com_vel1_world.y() = 0.0;
		com_vel1_world.normalize();

		const auto angle = __max(0.0, 1.0 - com_vel1_world.dot(goalDir));
		reward = exp(-2.5 * angle * angle);
	}

	return reward;
}

and
double cSceneImitate::CalcSimHeadingRewardGoal(const cSimCharacter& sim_char) const
{
	double reward = 0.0;
	auto ct_ctrl = dynamic_cast<cCtController *>(this->GetController().get());
	if (ct_ctrl != nullptr)
	{
		const auto goalDir = ct_ctrl->GetGoalDir();

		// Get the Center-Of-Mass velocity along the XZ plane only...
		tVector com_vel0_world = sim_char.CalcCOMVel();
		com_vel0_world.y() = 0.0;
		com_vel0_world.normalize();

		const auto angle = __max(0.0, 1.0 - com_vel0_world.dot(goalDir));
		reward = exp(-2.5 * angle * angle);
	}
	return reward;
}

The first calculates the heading reward for the kinematic actor. The latter do the same for the simulated one.
5] Implemented (in CalcReward) 3 methods to calculate the Multi-Clip premium:
Method 1:

double r = 0, bestReward = 0;
std::shared_ptr<cKinCharacter> mBestKin = nullptr;
for (auto &mKinChar : mKinChars)
{
    const double r2 = CalcKinHeadingRewardGoal(*sim_char, *mKinChar);
    if (r2 > bestReward)
    {
        bestReward = r2;
        mBestKin = mKinChar;
    }
}
r = CalcRewardImitate(*sim_char, *mBestKin) * 0.70 + CalcSimHeadingRewardGoal(*sim_char) * 0.30;
return r;

This method first selects the best kinematic actor based on its the heading reward (i.e. the actor mostly aligned with the heading direction vector). After choosing one, the final reward is given by the imitate reward (70%) and by the simulated actor's heading reward (30%).
Method 2:
double r = 0;
for (auto &mKinChar : mKinChars)
{
    const double r2 = CalcRewardImitate(*sim_char, *mKinChar) * 0.70 + CalcKinHeadingRewardGoal(*sim_char, *mKinChar) * 0.30;
    r = __max(r, r2);
}
return r;

The second method totally ignores the reward of the simulated actor. It calculates the reward for each kinematic actor considering its heading and imitate reward, then take the maximum.
Method 3:
double r = 0;
for (auto &mKinChar : mKinChars)
{
    const double r2 = CalcRewardImitate(*sim_char, *mKinChar) * 0.70 +
                                  CalcKinHeadingRewardGoal(*sim_char, *mKinChar) * 0.15 +
                                  CalcSimHeadingRewardGoal(*sim_char) * 0.15;
    r = __max(r, r2);
}
return r;

The latter method is similar to the second, but the simulated actor's heading reward is also added.
Ok, just to simplify things, I've fixed the heading direction (during training) to 45 degrees.
The 1st and 3rd methods give a result like the following video shows:
https://youtu.be/5XuMRXfK1os
As you can see, the actor deviates only a little but, above all, it doesn't walk correctly: it advances by doing consecutive mini jumps! I tried to tune the weighting factors, but these mini jumps don't disappear!
The 2nd method is more promising:
https://youtu.be/pJe0ab6uu3M
As you can see, this time the actor (almost) follows the set direction of 45 degrees, but it limps a bit (trained for 50 million samples).
So, none of these methods are completely satisfactory (and I haven't tested them yet by varying the heading vector during training). What is missing or I've misunderstood?
Can you please give me some good direction on how to properly model the reward function? Do I have to consider the heading reward of both kinematic and simulated actors? Or only one?
And maybe, with multiple clips, we have to train the actor for by far more samples?
Lastly, could the consecutive mini jumps exhibited by the 1st and 3rd method be due to a local maximum?
The questions are many, I know. Thanks for any reply.

Hi,
I am also trying to reproduce the multi-clip implementation.
Could you share how you loaded multiple clips and created several kinematic characters?


"Retrying dependency api due to error" issue

Hi all. I'm currently trying to do a bundle install for a particular project, but keep getting the error:
Retrying dependency api due to error (2/4): Bundler::HTTPError Network error while fetching
https://bundler.rubygems.org/api/v1/dependencies?gems=faraday%2Cactivesupport%2Crack%2Coogway_profile%2Cslim%2Cbskyb-global-clients%2Cmysky_logging%2Cslogger%2Cjson_pure%2Crubyzip%2Crest-client%2Cterminal-table%2Cinteract%2Caddressable%2Cuuidtools%2Crb-readline%2Ccaldecott-client%2Cescape_utils%2Coj%2Cjson%2Csinatra%2Csinatra-contrib%2Csprockets%2Csprockets-helpers%2Cbskyb-rack-middleware%2Ccustomer_overview%2Cether-config-mysky%2Crack-fiber_pool%2Ceventmachine%2Cem-synchrony%2Cem-http-request%2Credis%2Chiredis%2Credis-store%2Credis-rack%2Cnewrelic_rpm%2Cnewrelic-faraday%2Cthin%2Cnimbus%2Cvmc%2Chashie%2Cpage_magic%2Cpoltergeist%2Cchildprocess%2Cwait%2Cmirage%2Cnokogiri%2Ccapybara%2Csinatra-websocket%2Cwaitforit

However, if I try following that URL, it takes a few seconds then downloads a 'dependencies' file, so clearly I'm able to hit it. It's also not a transient network error as I did a bundle install on the particular project (A) and bundle install on another, unrelated project (B) at the same time - project A's bundle install failed but B's bundle install worked fine and didn't get these errors.
One thing that works is if I use the suffix --full-index - in that case, it works perfectly, although it would be nice to know why it's failing without. Is it timing out when finding dependencies because there are too many gems possibly?
I've tried running it on a different PC (two dev boxes and a deployment box) and get the same issue. Any ideas for what could be causing this? Using ruby-2.1.1 and bundler 1.11.2.

Can you share the output of bundle env so we can try and reproduce the issue?

Warning: this Gemfile contains multiple primary sources. Using `source` more than once without a block is a security risk, and may result in installing unexpected gems. To resolve this warning, use a block to indicate which gems should come from the secondary source. To upgrade this warning to an error, run `bundle config disable_multisource true`.
Your Gemfile lists the gem httparty (>= 0) more than once.
You should probably keep only one of them.
While it's not a problem now, it could cause errors if you change the version of just one of them later.
 Still getting this for some projects continually, but some seem to be exempt from this.

@segiddins is just me, or is our Postgres instance getting slower and slower? :/ it looks like we need to raise the statement timeout again

Don't suppose anyone has had a chance to look at this? It is causing a few issues now when we try and run bundler package and the --full-index workaround won't work on that package

@jamesRaybould I think you can run bundle install --full-index --freeze, then bundle cache, then bundle install --no-freeze. Maybe. :P

Running:
bundle install --full-index --frozen
bundle cache
bundle install --no-frozen

Still doesn't work and still errors out on the cache stage:
Updating files in vendor/cache
 * rake-10.3.2.gem
 * i18n-0.7.0.gem
Fetching gem metadata from http://gem-server.internal.server/....
Fetching gem metadata from https://rubygems.org/.Retrying dependency api due to error (2/4): Bundler::HTTPError Network error while fetching https://bundler.rubygems.org/api/v1/...

Again, seems to be an issue only when we're using the vmc gem. We've raised the issue with VMC here, but they haven't responded yet. The issue is still happening though.

I hit this error just now, it has something to do with my gem server geminabox.  Disabling one of its configuration lines fixed it, though is cumbersome due to having to manually push things.  maybe we've miconfigured geminabox somehow?
Note the freshly commented out line in my config.ru file:
require "rubygems"
require "geminabox"

Geminabox.data = "/media/john/lijfdk/storage/geminabox_store/data"

# Geminabox.rubygems_proxy = true

run Geminabox::Server

I wonder if manually pushing vmc to your internal gem server will resolve this (or lead to an error on the next gem down that isn't already cached on your private gem server).

Are there any action items here? It seems VMC is no longer maintained.
Additionally, is it reproducible/still an issue with the latest version of bundler? (v1.12.4)

I would chalk this up to a problem with geminabox, close the ticket and see if @tmharber comes back with fresh information.


TemplateError: Only user and assistant roles are supported!

RuntimeError: Unsloth: Your prompt template must have 2 examples showing the user input {INPUT} and the assistant output {OUTPUT}

For example what is not allowed is just:
### Input:\n{INPUT}\n\n### Response:\n{OUTPUT}\n

What is required is 2x of this:
### Input:\n{INPUT}\n\n### Response:\n{OUTPUT}\n### Input:\n{INPUT}\n\n### Response:\n{OUTPUT}\n

From understanding the above Error, I composed (below) this chat_template  as mentioned (above) wrt Unsolth :
chat_template = """Below describes a series of conversations where a user expresses interest in various academic programs and provides their educational background.

The assistant responds with the name of the university associated with each program.

### Input:
User expresses interest in a Master program in Artificial Intelligence.

### Response:
Stanford University

### Input:
User expresses interest in a PhD program in Environmental Science.

### Response:
University of California, Berkeley

>>> User Educational Interest:

{INPUT}

>>> University Name:

{OUTPUT}

>>> Educational Background:

{INPUT}

>>> University Name:

{OUTPUT}

>>> Educational Background:

"""

While creating a dataset with apply_chat_template
dataset = apply_chat_template(
    dataset,
    tokenizer=tokenizer,
    chat_template=chat_template,
)

Error:
TemplateError: Only user and assistant roles are supported!

Could you provide some examples of your dataset? this can help to reproduce this issues

Could you provide some examples of your dataset? this can help to reproduce this issues

Yes, It looks like this:

university
course
title
country
stream
duration
score_1
score_2
score_3
score_4

university_1
course_A
title_A
country_A
stream_A
duration_x
scores
scores
scores
scores

university_2
course_B
title_B
country_B
stream_B
duration_Y
scores
scores
scores
scores

university_3
course_C
title_C
country_C
stream_C
duration_Z
scores
scores
scores
scores

university_4
course_D
title_D
country_D
stream_D
duration_X
scores
scores
scores
scores

university_5
course_E
title_E
country_E
stream_E
duration_Y
scores
scores
scores
scores

score_1 is in float and score_2. score_3, score_4 are in int.
rest cols are in int

Having the same issue here

I use unsloth.to_sharegpt to merged_prompt, after I change "from" to "role" for below two lines, It works, hope helpful.
https://github.com/unslothai/unsloth/blob/d91d40a7b6b556f2d1fdd3e1e430f7a76a799627/unsloth/chat_templates.py#L1071-L1072
Reference: This is the demo of tokenizer.apply_chat_template

Final code
chat_template = """Below describes a series of conversations where a user expresses interest in various academic programs and provides their educational background.

The assistant responds with the name of the university associated with each program.

### Input:
User expresses interest in a Master program in Artificial Intelligence.

### Response:
Stanford University

### Input:
User expresses interest in a PhD program in Environmental Science.

### Response:
University of California, Berkeley

>>> User Educational Interest:

{INPUT}

>>> University Name:

{OUTPUT}

>>> Educational Background:

{INPUT}

>>> University Name:

{OUTPUT}

>>> Educational Background:

"""

dataset = load_dataset('csv', data_files="./drive/MyDrive/data/unsloth-issues-994.csv", split = 'train')
dataset = dataset.rename_column("score_1", "output")

from unsloth.chat_templates import apply_chat_template, _parse_combined_prompt, _create_formatter

def to_sharegpt_update(
    dataset,
    merged_prompt = "",
    merged_column_name = "instruction",
    output_column_name = "output",
    remove_unsued_columns = True,
    conversation_extension = 1,
    random_state = 3407,
):
    """
    Converts a dataset to ShareGPT style.
    ShareGPT requires only 1 input and 1 output field.
    This means one has to merge multiple columns into 1 for 1 input field.
    Use `conversation_extension` to increase the length of each conversation by randomnly
    selecting a few and packing them into 1.

    merged_prompt = "",                 Prompt to merge columns into 1 input
    merged_column_name = "instruction", Final column name for the input  field
    output_column_name = "output",      Final column name for the output field
    remove_unsued_columns = True,
    conversation_extension = 1,         Automatically combines `conversation_extension` convos into 1
    random_state = 3407,
    """
    if "conversations" in dataset.column_names:
        convo = dataset[0]["conversations"]
        if type(convo) is list:
            raise TypeError("Unsloth: Your dataset is probably already in ShareGPT format!")
        pass
    pass

    possible_columns, final_optional_prompts = _parse_combined_prompt(merged_prompt, dataset)
    function = _create_formatter(possible_columns, final_optional_prompts, merged_column_name)
    exec(function, globals())
    dataset = dataset.map(__combined_prompt_processor__, batched = True, desc = "Merging columns")

    def __convert_to_sharegpt__(examples):
        users      = examples[merged_column_name]
        assistants = examples[output_column_name]
        texts = [
            [
                {"role" : "user",      "content" : str(user)     }, # update from -> role
                {"role" : "assistant", "content" : str(assistant)}, # update from -> role
            ] \
            for user, assistant in zip(users, assistants)
        ]
        return { "conversations" : texts, }
    pass

    dataset = dataset.map(
        __convert_to_sharegpt__,
        batched = True,
        desc = "Converting to ShareGPT",
        # Remove unsued columns!
        remove_columns = dataset.column_names if remove_unsued_columns else None,
    )

    # Randomnly concat conversations to create a long stream!
    from datasets import concatenate_datasets
    n_extensions = max(conversation_extension-1, 0)
    if n_extensions == 0: return dataset

    dataset = dataset.rename_columns({"conversations" : f"conversations0"})
    all_shuffled = [dataset]
    for j in range(1, n_extensions+1):
        shuffled = dataset.shuffle(seed = random_state+j).rename_columns({"conversations0" : f"conversations{j}"})
        all_shuffled.append(shuffled)
    pass
    dataset = concatenate_datasets(all_shuffled, axis = 1)

    # Combine them into 1
    function = "def __combine_conversations__(examples):\n"
    n_extensions += 1
    for j in range(n_extensions):
        function += f"{' '*4}conversations{j}__ = examples['conversations{j}']\n"
    function += f"{' '*4}convos = []\n"
    function += f"{' '*4}for ({', '.join(f'conversations{j}' for j in range(n_extensions))}) "\
                f"in zip({', '.join(f'conversations{j}__' for j in range(n_extensions))}):\n"
    function += f"{' '*8}convos.append("\
                f"{'+'.join(f'conversations{j}' for j in range(n_extensions))})\n"
    function += f"{' '*4}return " + "{ " + f"'conversations' : convos" + " }"

    # Map function
    exec(function, globals())
    dataset = dataset.map(
        __combine_conversations__,
        batched = True,
        desc = "Extending conversations",
        # Remove unsued columns!
        remove_columns = dataset.column_names if remove_unsued_columns else None,
    )
    return dataset
pass

merged_prompt_val = 'merge {university}, {course}, {title}, {country}, {stream}, {duration} as instruction'
dataset = to_sharegpt_update(dataset, merged_prompt = merged_prompt_val)

I don't know if it is suitable for other situation, I will try to submit a PR.

Solved , Thanks !!


Something goes wrong to fetch simcards: getPhoneNumberFromFirstAvailableSource: Neither user 10705 nor current process has android.permission.READ_PHONE_NUMBERS or android.permission.READ_PRIVILEGED_PHONE_STATE or carrier privileges

i'm gettig this error while doing both getSimCards and getSimCardsNative in android
please help on this
Something goes wrong to fetch simcards: getPhoneNumberFromFirstAvailableSource: Neither user 10705 nor current process has android.permission.READ_PHONE_NUMBERS or android.permission.READ_PRIVILEGED_PHONE_STATE or carrier privileges

Linked to #60


How to turn off hibernate function in rancher OS?

RancherOS Version: (ros os version)
0.7.1
Where are you running RancherOS? (docker-machine, AWS, GCE, baremetal, etc.)
Laptop
Hi. I am using laptop to run rancher os, and it's pretty good server for me :)
But, i got a problem little bit about hibernate/sleep problem.
If i close my laptop just like this, to turn off just monitor

rancher os is going to be hibernated. and what i want is to turn off this function.
All i need is to turn off mornitor, which is not nessessary for server!
Is there any way to do it? thanks :D

It's built into the acpid container.  Lid close event will trigger a hibernate.  You can see how this works here https://github.com/rancher/os/blob/master/images/02-acpid/Dockerfile
I would recommend using write_files to overwrite suspend.sh with exit 0 http://docs.rancher.com/os/configuration/write-files/

I'll check it out :D
Thank you ibuildthecloud.

sudo ros s stop acpid
sudo ros s rm acpid

@Albus stopping and removing the acpid service did not work in my case.  When closing the lid, the system went to sleep.
The method I'm using now is to set acpi=off in syslinux.  Unfortunately, this does not persist across a rancherOS upgrade (but does persist across reboots).
sudo ros config syslinux
acpi=off


[Lazy Loading] Wait for all images to be loaded to get all the src URL not the data:image/gif;base64...==

I am scraping a page with 500 items but i can't get all the URL of the images because  some of them (the majority) are not loaded before the browser is closed. Some src attributes have their correct URL but for those that are not loaded i got data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==
Here is part of my code, but i got AttributeError: 'ScrapyPlaywrightDownloadHandler' object has no attribute 'contexts'
when i write PageCoroutine("wait_for_timeout", 120000)
    def start_requests(self):
        yield scrapy.Request(
            url=self.casami_url_list_product,
            # meta={"playwright": True},
            meta=dict(playwright=True,
                      playwright_include_page=True,
                      playwright_page_coroutines=[
                          PageCoroutine(
                              'wait_for_selector', "//div[contains(@class, 'products-list')]"),
                          PageCoroutine("evaluate", "window.scrollBy(0, document.body.scrollHeight)"),
                          PageCoroutine("wait_for_timeout", 120000),  # got AttributeError: 'ScrapyPlaywrightDownloadHandler' object has no attribute 'contexts' when declared
                          PageCoroutine("screenshot", path=Path(__file__).parent / "recovered.png", full_page=True)
                      ])
        )

    # async def parse(self, response):
    def parse(self, response):
        all_product_items = response.xpath("//div[contains(@class, 'products-list')]"
                                           "//div[contains(@class, 'product-layout product-grid')]")
        product_name_xpath = "//div[contains(@class, 'product-image-container')]/a/@title"
        product_image_url_xpath = "//div[contains(@class, 'product-image-container')]//img/@src"
        product_price_xpath = "//div[contains(@class, 'price')]/span[contains(@class,'price-new')]/text()"
        product_detail_url_xpath = "//div[contains(@class, 'product-image-container')]/a/@href"

        for product_item in all_product_items:

            yield {"name": product_item.xpath("." + product_name_xpath).get(),
                   "price": product_item.xpath("." + product_price_xpath).get(),
                   "image_url": product_item.xpath("." + product_image_url_xpath).get(),
                   "product_detail_url": product_item.xpath("." + product_detail_url_xpath).get()}

Thanks for reply

You've encountered a data URL, which in this case happens to be a base64-encoded GIF image. That is a valid image, and it will be displayed by browsers as is.
See also:

https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URIs
https://stackoverflow.com/a/9967193


Add support for a wider variety of EnOcean devices

Breaking Change:

Theoretically, nothing should break. The default configuration is compatible with previous versions of the component. However, since I do not own all of the devices, I cannot guarantee it. I suspect that previous commiters @rubund and @NoUsername own some of the devices. Guys, if you read this, I would be very thankful if you could spend the time to review and test this PR.
Tested with:

2 Rocker Switch
Temperature Sensor

Description:
This PR adds support for temperature sensors and humidity sensors. The telegram message is now closer to the officially documented messages, which in theory should support a wider variety of devices.
As a result of the new sensors and the more generic message parsing, a refactoring was necessary. The enocean package was updated, since newer versions can do the parsing for us, moving some complexity out of the component.
The tests should make sure that the component still works for existing devices.
Note: I am away on an one week vacation, starting on the 16th. So take your time reviewing it 😃
Related issue (if applicable): none
Pull request in home-assistant.io with documentation (if applicable): home-assistant/home-assistant.io#8939
Example entry for configuration.yaml (if applicable):
sensor:
  - platform: enocean
    id: [0x20, 0x20, 0x20, 0x20]
    name: Sensor 1
    device_class: temperature
  - platform: enocean
    id: [0x20, 0x20, 0x20, 0x20]
    name: Sensor 2
    device_class: humidity
  - platform: enocean
    id: [0x20, 0x20, 0x20, 0x20]
    name: Sensor 2
    device_class: powersensor

Checklist:

[x] The code change is tested and works locally.
[x] Local tests pass with tox. Your PR cannot be merged unless tests pass
[x] There is no commented out code in this PR.

If user exposed functionality or configuration variables are added/changed:

[x] Documentation added/updated in home-assistant.io

If the code communicates with devices, web services, or third-party tools:

[x] New dependencies have been added to the REQUIREMENTS variable (example).
[x] New dependencies are only imported inside functions that use them (example).
[x] New or updated dependencies have been added to requirements_all.txt by running script/gen_requirements_all.py.
[x] New files were added to .coveragerc.

If the code does not interact with devices:

[x] Tests have been added to verify that the new code works.

I am back and the test errors (doh!) are fixed now

Thanks @MartinHjelmare for the reviews. I fixed everything (I believe).
As discussed in Discord

We keep the non-breaking
The tests are removed and will go in a new PR
The binary_sensor's event remains unchanged
The binary_sensor and the enocean component in general need refactoring

Hi, i create a pull request with a mix off your code and another pull request :
https://github.com/home-assistant/home-assistant/pull/23326
See if that's helping or not.

Please don't squash commits after review has started. Sqaushing commits makes it harder for readers to track changes.

Good job on the code!


Add namespacing for dbt resources

Placeholder issue for thoughts related to namespacing. When we're able to prioritize work on namespacing, let's convert this to a more actionable issue:
Questions:

how can we namespace models to avoid ambiguity?
what do package namespaces look like for models?
is there a way to "import" resources from a package, rather than injecting everything into the toplevel context?

Other similar questions / use cases welcomed :)

Hey @drewbanin, tagging looks like it'll do the trick. thanks fo the tip!

Hi @drewbanin
Is this "Help Wanted" still current?  I'm interested in supporting this feature in our internal projects because we have 100s of sql files that have a hierarchy of subdirectories under the "models" directory to help keep things semantically organized.  Also, we have several SQL files with the same name but that exist in different directories (as illustrated above with the sales.sql example).
Offhand, I think that this feature needs to be enabled with a flag somewhere to avoid getting an error during the compile phase when two SQL files have the same name (i.e., the "dbt found two resources with the name" CompilationException.  One place to add this flag would be in dbt_project.yml at the top-level.  Maybe like this:
dbt_project.yml:
  use_reference_namespace = true

The next trick is to thread this parameter through to the places where the unique identifiers are generated.  As one example, in parser/base_sql.py BaseSqlParser.parse_sql_node, a unique_id is generated by joining the resource_type, package_name, and node name (i.e., filename base) using '.'.  This unique_id is later split apart in utils.py id_matches().  As such, I don't think that we can use the '.' character as the separate the namespace.  Instead, maybe it could be a '/', just like the directory separator:
select * from {{ ref('shopify_uk/sales') }}
union all
select * from {{ ref('shopify_fr/sales') }}

So, I think one solution would approximately be to update the get_path methods (in parser/schemas.py and parser/base.py) to take a node instead of name as the last parameter, and then conditionally include the directory prefix in the name based on the use_reference_namespace flag.  All these directories would be relative to the directory specified in dbt_project.yml, and could include subdirectories of subdirectories (e.,g, ref('shopify_uk/sub_dir/sales')).
Thoughts?  Thanks!

(Actually, in looking closer at the id_matches() logic, there is support for using '.' in the name.  For example, NodeType.Source requires that there be a single '.' in the node_name)

Hey @heisencoder - before we begin any work here, I'd like to better develop our thinking around how namespacing will work in general. In your example (shopify_uk/sales), what is shopify_uk relative to? One obvious answer is the "root" models/ directory, but that doesn't extend super well to a world with packages.
dbt already has a notion of "fully qualified names" -- this looks like package_name.path.to.model_name. It's how the --models selector works, and how models are configured in the dbt_project.yml file. In dbt's selector logic, we have some code to assume that the package_name is the "root" package if one is not provided.
If namespacing is turned on, should it be allowed to ref a model without namespacing it? Or are namespaces solely intended to make it possible to handle models with shared names?
Presently, dbt will assert that no two models share the same name. I think we'll need to defer that validation until all of the refs are evaluated. If a ref can evaluate to more than one model, then we can make that a compile-time error.
I'm definitely more in favor of using dots (.) to represent namespaces compared to slashes (/) just insofar as these won't be "real" file paths.
What do you think about all of this?

Thanks @drewbanin for your thoughts!
I haven't looked into the "fully qualified names" that you've mentioned for the --models selector, so don't know how this proposal plays into that.  Maybe it can just slide in with minimal changes?  The package_name is already included in the node's unique_id, so maybe they can work together?
Here's a proposed approach that hopefully fits in the framework you've outlined:

Update the name computation to use the full path relative to the resources-path in which the model was found.  Use '.' to separate path components.  e.g. A file in 'models/dir1/dir2/model.sql' would have a name of 'dir1.dir2.model'.
Update utils.find_in_list_by_name and utils.find_in_subgraph_by_name to first attempt to find an exact match, and then later default to looking only at the model name without the package prefix to match current behavior.  However, If looking just at the model name, it will be necessary to find all such matches and raise an exception if two or more matches apply.
Hunt down and update any code that relies on the previous naming convention.

We could do these changes without adding a namespacing mode, although adding a namespacing mode could help with forward compatibility issues with the manifest.json format.
As a proof-of-concept, here are examples of the changes I imagined to utils.find_in_list_by_name:
def find_in_list_by_name(haystack, target_name, target_package, nodetype):
    """Find an entry in the given list by name.

    target_name could either be the base of the model name or could include the
    package prefix separated with '.' characters.
    """
    base_name_matches = []
    for model in haystack:
        name = model.get('unique_id')
        if id_matches(name, target_name, target_package, nodetype, model):
            return model
        if id_matches(name, target_name.split('.')[-1], target_package, nodetype, model):
            base_name_matches.append(model)
    if len(base_name_matches >= 2):
        raise_compiler_error(
            'Ambiguous reference %s has multiple matches: %s' % target_name, base_name_matches)
    if len(base_name_matches == 1):
        return base_name_matches[0]
    return None

(And then also update id_matches to not blow up when the target_name contains '.')
Note that as a side-effect of including the full package prefix in the target_name, the compile tests in _load_schema_tests no longer fail because they're using the fully qualified name.

Hey @heisencoder - I think everything you're saying here is totally reasonable. The big challenge I can imagine is that we've always held as a hard constraint that resource names are unique. As such, there may be pernicious failures in parts of the codebase which rely on these model names being unique. We'll definitely have to figure out how to adjust the unique_id attribute of nodes, as it would be very bad if those unique ids were no longer unique :)
I'm supportive of this idea, but we need to figure out:

how to generate unique ids for nodes (we can probably leverage the FQN path here)
understand the breadth of changes outside of the ref function context that are required:

i know the docs site will need to be updated
i don't believe we parse values out of the unique_id string anywhere, but we'd need to verify that
figure out things around the edges, like how schema.yml works when models can have duplicate names

This is probably going to be a pretty big project!

Given the large number of edge cases, I think it makes sense to flag control this new code so that old code doesn't have to worry about these edge cases.  So I'm thinking that this could be controlled by a use_namespaces = true option at the top-level of the dbt_project.yml file.  We could theoretically allow turning this off or on at different levels of the models section of the dbt_project.yml, but I'm hoping to avoid allowing that level of configuration.
The unique_ids for the nodes would then become the FQN path instead of just the base filename.  This should make them be unique in all contexts.
I can provide some suggested documentation text.
I hadn't thought about the name parameter in the schema.yml files, but here are my two thoughts:

The path prefix to the model name can be implied by the path prefix of the schema.yml file.  I'm guessing that in general, each directory will have its own schema.yml file that describes the models in the same directory.  Since this is convention, I'm guessing that this won't always be followed.
We could expand the syntax to allow the models.name field to include the dotted FQN to the model name.  This would allow schema.yml files to describe models in other directories.

Why not just copy what other programming environments do (python) and make the folder the package name? As a newbie, I expected it already worked list this. I think this should be the default with an option to disable it that allows legacy users to migrate.

Also, I will add that the source selector works about like I'd want from a usage point of view. That may provide a better way to migrate as well. Any time the old ref() is called, the uniqueness for that model name is ensured, and generates and exception if there are duplicates. A new ref that uses packages (or maybe just a new arg!) would limit this check to the namespace that was specified.

I love the source('my_schema', 'my_table') syntax. To echo @preston-hf, could we replicate that syntax for ref with ref('my_schema', 'my_model')? If the schema name is set in schema.yml or dbt_project.yml, then that is the model's schema/namespace. If not set, then we keep the current behavior.

hey @rpedela-recurly - we definitely don't want to namespace things by schema name! Destination schemas are configuration in dbt, so the actual schema that a model renders into should be totally different in dev vs. prod.
You can actually currently supply two arguments to the ref function, eg:
{{ ref('package_name', 'model_name') }}

In practice, this isn't super useful though -- model names must currently be unique across all packages included in a project. Increasingly, my thinking is that we should:

make it easier to define multiple packages inside of a single project
drop the requirement that model names must be unique across packages

If two models share the same name, then you would need to qualify the model name with a package name in the ref statement. If the two models have the same database representation, then dbt could raise a compiler error.
Do you buy that?

I didn't realize you could define package_name. Number 1 would suffice I think then, and at least I would make package_name = schema_name, but there would be more flexibility for others. As far as defining package/model relations, I still like setting the package for a set of models in either dbt_project.yml or schema.yml. But I also like having the ability to keep packages the way they are, as I see them being useful for large projects with many contributors. What if ref was updated to allow this:
ref('package', 'namespace_within_package', 'model')
ref('namespace_within_project', 'model')

Then namespace is defined in schema.yml or dbt_project.yml. I really like having the source/ref syntax look like SQL schema_name.table_name even if the real schema differs because of dev/prod environments.

@drewbanin I like the two notions of making packages within a project easy, and dropping the name uniqueness across packages.
This is similar to ideas that Betterment has been tossing around a lot lately. We have models across a small but slowly growing number of internal domains that all belong in the same project, but have different owners and use cases depending on the domain. We want to relax the requirement for unique model names across these domains (especially for ephemeral models).
We were thinking of suggesting custom schema names (not the actual materialized schema names, which may differ from run to run) as a namespace, but I think if we could easily create intra-project packages, that could suffice. The requirements that we've been thinking about are as follows (let's say for these requirements that namespace could either be the name of a package or of a custom schema):

Models can be identified either by "model_name" or "namespace", "model_name".
e.g., If two models have the same name x but different namespaces A and B:

DBT does not fail to compile the project, as it does now
Using ref('x') from a model in a different namespace (not A or B) fails in compile because the reference is ambiguous
Using ref('A', 'x') from. a model in a different namespace than A is fine
Using ref('x') from a model in namespace A implies ref('A', 'x') (i.e., DBT first it checks whether x is unambiguous and, if it is not, tries to find 'A', 'x')

I know everyone on our team would also be thrilled if ephemeral models within different folders of the same package could be similarly named without creating conflicts as well, but the points above are more important.

Also @drewbanin, when you say "make it easier to define multiple packages inside of a single project", what do you imagine that looks like? Something along the lines of the following structure, where DBT perhaps infers the package names of sub-projects, and maybe inherits project config from the "parent"?:
project/
|-- domain1/
|   |-- models/
|   +-- tests/
|
|-- domain2/
|   |-- models/
|   +-- tests/
|
|-- models/
|
|-- tests/
|
|-- dbt_project.yml
+-- packages.yml

# project/packages.yml

packages:
  - local: domain1
  - local: domain2

# project/dbt_project.yml

name: 'company_project'
version: '1.0'

models:
  domain1:
    schema: d1

  domain2:
    schema: d2

I am very interested in being able to namespace objects like models and sources and macros, at least in a rudimentary fashion by removing the requirement that model names be unique even between packages, and I'd be happy to try to put together a PR if someone more familiar with the project can give me a little guidance on where to get started.

hey @mjumbewu - thanks for writing all of this out!
I really like your suggestions on how we should approach model-level namespacing: it's really sensible and well-specified.
Re: making namespacing easier: I like the general idea of your suggestion:
packages:
  - local: domain1
  - local: domain2

The only issue with this approach is that you need to run dbt deps to create symlinks from dbt_modules/domain1/ to domain1/. This would be a little bit circuitous and it doesn't work incredibly well on Windows (you know, no symlinks).
Instead, I think we could do one of the following:
1. Smarter local packages
We could add a config like symlink to the packages.yml file for local: packages. If this value is false, then dbt wouldn't try to symlink the package into dbt_modules/ and would instead point to the package dir directly.
Pros:

Feels pretty natural
You could use arbitrary file paths

Cons:

I think this would be a big change to dbt. We definitely can read the packages.yml file and use it to inform parsing/running, but I don't think that's how things work today

2. A new dbt_project.yml config
The solution to having too many configs is to add another config. We could add a top-level dbt_project.yml config like package-paths. This config would let you enumerate paths in your project that contain dbt packages. Given the example you showed above, that would look like:
project/
|-- domain1/
|   |-- models/
|   +-- tests/
|
|-- domain2/
|   |-- models/
|   +-- tests/
|
|-- models/
|
|-- tests/
|
|-- dbt_project.yml
+-- packages.yml

# dbt_project.yml

source-paths: ['models']
package-paths: ['domain1', 'domain2']

Alternatively, we could get clever with source-paths and support something like:
source-paths:
  project: ['models']
  packages: ['domain1', 'domain2']

But I don't think I love that approach.
Pros:

I think this should be relatively straightforward to implement today. dbt already knows to look in dbt_modules for packages, and this would be an extension to that logic

Cons:

???
I can't think of any real downsides to this approach

3. config-level namespaces
This was the one I had in mind when I wrote about making namespacing easier, but increasingly, I don't think this approach is such a good idea. I was picturing a models: level config which supports a namespace as a config value. That would look like:
models:
  my_project:
    path1:
      namespace: domain1
    path2:
      namespace: domain2

All of the models in models/path1 would be namespaced under the package domain1, and all of the models in models/path2 would be namespaced under the package domain2.
Pros:

It's pretty flexible!
Models/maros/seeds/snapshots/etc can live closer together, rather than in totally separate folders
Cons:
I think an approach like this would make correctly parsing models pretty challenging!
The models: config is only intended to configure models, but we'd want to namespace other resources (say, tests/snapshots/seeds/etc) in a package-aware way too

These are the things bouncing around in my head. I think approach 2 outlined above is going to be our best bet, but I'm curious what you all think too.

I like #2 the best.

Hi there, just wanted to continue that discussion
I'm a beginner on DBT, so I would just like to give my opinion from a "beginner" point of view.
Seeds
First I would like to point out that this would be useful for Seeds as well, consider the following simple use-case:
2 pipelines are being worked on, they target 2 different databases, A and B, this is the folder structure
data/a/calendar.csv
data/b/calendar.csv

The 2 pipelines target 2 different Databases, so their names do not clash when they run against the Warehouse. But A needs a different fiscal calendar than B.
Now, as a beginner, my first intuition would be to use ref("my_proj", "a.calendar") because of the --select argument available to dbt seed
Sub projects
I think the "sub project" abstraction is quite hard to wrap my head around:

Now I need to look at the config to figure out if a directory has a special meaning?
What if I have a name clash in a large sub-project? Aren't we back to this issue?

Proposal
(I'm aware it's a summary/mix of some other solutions above)

Have the ability to call ref-able objects either by their full "module path" ref("my_proj", "a.calendar") or their name "excluding module" ref("my_proj", "calendar"), note that ref("a.calendar") or ref("calendar") should be valid if it's not ambiguous cross project

In the function to resolve the object, we would look at a.calendar and see that there aren't duplicate objects returned, in case calendar is used, we could return an Exception "Ambiguous ref: use either a.calendar or b.calendar" . I'm actually not familiar with how the code deals with ambiguous objects cross packages but I would imagine part of the logic could be re-used.

I'm not very experienced with the generated documentation, maybe we could display names of objects as full explicit references in case of ambiguity (perhaps by running a version of the above logic on all objects to be rendered), or just the object name otherwise. Again I'm not familiar with how ambiguous objects cross packages are rendered on the docs but it could inform implementation.

Pros:

Simplicity
No config change
Intuitive to beginners
Shielded against potential clashes of the sub-projects solution

Cons:

Can't assume uniqueness within a project
Need to introduce logic to detect ambiguous objects in modules

@drewbanin Where does this go from here? Is this the 'DBT extension proposal'? Do you need to make a BDFL pronouncement?
Was shocked to discover that this is an outstanding issue still. I've always used multiple packages with qualified refs and I guess just got lucky that I never repeated a model name.

@drewbanin
Hi Drew,
I was horrified when I just noticed that I could not just namespace the same model name by putting these in the different packages. When one references with {{ ref('package1', 'model1') }}and {{ ref('package2', 'model1') }}  I definitely assumed that it would indeed be valid to include these in the same top-level dbt project without a clash.
Instead of discussing more advanced ways (as in bigger changes) of namespacing as above, would it be possible to at least make this approach work, it seems to do the trick (for me at least ) ? I would think this makes for less surprises when using the dbt-framework. Is there anything in the way that dbt is structured internally that makes a change like this painful?
I haven't looked into the source code in dbt yet, but I might be able to contribute with a PR if you find that this would be a suitable way forward.
And sorry for whining :) dbt is a great tool, thanks for your contributions!

oh geez... I missed this one and am disappointed that I have abdicated my BDFL responsibilities. Thanks for the bump @hhagblom and for getting this back on my radar!
Yep.... it is highly weird that dbt supports a two-arg version of ref, but still requires model names to be unique across packages. I'd need to think hard about what kind of downstream effects will come out of lifting the "model names are unique" constraint that's been in place for the entirety of dbt's development, but here's an optimistic scenario:

we use a unique_id everywhere and the impacts of making model names non-unique across packages in minimal
we remove the validation that ensures that model/node names are globally unique, and instead only require uniqueness within a project
we keep the validation which ensures that two nodes do not point to the same database.schema.identifier in the database

@jtcohen6 curious to get your take on this one too. Will poke around here and report back with findings :)

ok @hhagblom & co - let's not make a habit out of this, but I did open my first PR to dbt in a couple of months over here https://github.com/fishtown-analytics/dbt/pull/3053/files
I'm going to need to spend more time here with testing, but if any of y'all want to take this for a spin locally, feedback is very welcomed :D

ok @hhagblom & co - let's not make a habit out of this, but I did open my first PR to dbt in a couple of months over here https://github.com/fishtown-analytics/dbt/pull/3053/files
I'm going to need to spend more time here with testing, but if any of y'all want to take this for a spin locally, feedback is very welcomed :D

@drewbanin Many thanks! What I'll do is that I'll fork the repo and will do a rebase-onto a stable release and take this for a spin in our project. I'll let you know how it works out for us!

ok @hhagblom & co - let's not make a habit out of this, but I did open my first PR to dbt in a couple of months over here https://github.com/fishtown-analytics/dbt/pull/3053/files
I'm going to need to spend more time here with testing, but if any of y'all want to take this for a spin locally, feedback is very welcomed :D

@drewbanin Many thanks! What I'll do is that I'll fork the repo and will do a rebase-onto a stable release and take this for a spin in our project. I'll let you know how it works out for us!

We have tried this out, although it works well building the model, there is a drawback and that is that the model names in the model definitions seem to clash, so if we have two .yml files with the same model declaration in them in different packages (referred from the same top-package), then these still seem to clash.
I would be willing to have a further look on how to fix it, although I have a deadline coming up so I mightn't get to it in the near future.
If there is an easy fix for it (I haven't investigated the dbt-code) I'd be happy try out another build and provide feedback.

Hello folks. Any updates on this issue?

@Shadowsong27 The work in #3053 got 90% of the way to supporting this, but it ran into an obstacle around defining model properties in .yml files, since each set of properties is associated with a model name only. While I haven't had a chance to confirm, I believe some of the in-progress changes to file parsing (#3364) may incidentally resolve this by storing a fuller reference and changing the way that dbt adds properties to model nodes. So I'd like to revisit this once v0.20 is out the door.

Sorry for the +1 🙈  but we just ran into this as well. We're moving through different layers, raw -> clean -> common, where the models are refined in each layer. Currently we have to name the models differently coming from each layer to prevent the name collision.

Is there any news on the status of this issue? This and unit testing are the two big ones for me to make dbt feel like the complete real deal :)

@louis-vines This is still very much top-of-mind for me. I don't have any immediate plans for development that I can share, but I am thinking that this will be an important ingredient for supporting dbt deployments at larger orgs. Probably part of a larger capability around cross-project lineage, which could exist without this capability, but would be much much more desirable with it.

For folks following this issue, and interested in namespacing capabilities as a crucial tool for scaling out dbt deployments: I'd be very curious to hear your thoughts on the proposal outlined in https://github.com/dbt-labs/dbt-core/discussions/5244

How does anyone use dbt at scale without this? My team was hoping to switch over to dbt but without some form of namespacing I don't think we can afford the friction in adjusting our database and workflow!

By prefixing every models?

How does anyone use dbt at scale without this? My team was hoping to switch over to dbt but without some form of namespacing I don't think we can afford the friction in adjusting our database and workflow!

Alternatives:

If you need to repeat table names in different layers maybe you can use different projects. (Not ideal)
You can control the table name using alias. You have to prefix the .sql files but you can control the table name: https://docs.getdbt.com/docs/building-a-dbt-project/building-models/using-custom-aliases
This can be auto generated by a macro.

How does anyone use dbt at scale without this? My team was hoping to switch over to dbt but without some form of namespacing I don't think we can afford the friction in adjusting our database and workflow!

Alternatives:
1. If you need to repeat table names in different layers maybe you can use different projects. (Not ideal)

2. You can control the table name using alias. You have to prefix the .sql files but you can control the table name: https://docs.getdbt.com/docs/building-a-dbt-project/building-models/using-custom-aliases
   This can be auto generated by a macro.

Oh, I rather like the second option. Thank you for the suggestion @hernanparra.

How does anyone use dbt at scale without this? My team was hoping to switch over to dbt but without some form of namespacing I don't think we can afford the friction in adjusting our database and workflow!

Alternatives:

If you need to repeat table names in different layers maybe you can use different projects. (Not ideal)
You can control the table name using alias. You have to prefix the .sql files but you can control the table name: https://docs.getdbt.com/docs/building-a-dbt-project/building-models/using-custom-aliases
This can be auto generated by a macro.

I'm not sure I understand point 2. I set up a sample project like
project/
|-- models
|   |-- example
|       |-- my_first_dbt_model.sql
|       |-- my_second_dbt_model.sql
|
|   |-- folder2
|       |-- my_first_dbt_model.sql

where folder2/my_first_dbt_model.sql is
-- My other first model
{{ config(alias='different_model_name') }}
select 1

This still fails because I have two models with the same name even though I wanted to alias one of the models to a different identifier.
19:28:01  Running with dbt=1.1.2
19:28:01  Encountered an error:
Compilation Error
  dbt found two models with the name "my_first_dbt_model".

  Since these resources have the same name, dbt will be unable to find the correct resource
  when looking for ref("my_first_dbt_model").

  To fix this, change the name of one of these resources:
  - model.dbt_sample.my_first_dbt_model (models/folder2/my_first_dbt_model.sql)
  - model.dbt_sample.my_first_dbt_model (models/example/my_first_dbt_model.sql)

@stevenlw-porpoise I think the suggestion is to use alias in the opposite direction, e.g. call your file different_model_name.sql but use the alias to rename it back to my_first_dbt_model via the alias. It's not a great workaround, as you still need to reference different_model_name in refs which makes migration harder.
This is a bit of a hassle for us migrating some dataform pipelines over (one of the few areas that dataform is better), but we do already break things into a bunch of different pipelines (which has its own problems) which mitigates the problem somewhat.

@elyobo Tried that but DBT doesn't let me create it. Model name has to be unique both in custom alias and file name for same schema and database.

This is one of the most significant dbt inconveniences. It would be nice to see this issue progress this year, it's been three years already.

There is some promising looking work as part of the broader cross project work - https://github.com/dbt-labs/dbt-core/discussions/5244

Maybe I am late to the party, but just like there are macros for generating a schema and a database name, why isn't there a macro to generate the model name?
I don't know how complicated it would be to add this feature, but it sure seems like overall, it would be the simplest approach. The previous behavior would be kept as default, not breaking anything and people that wanted to, can customize it to their will.
Currently, my team has a structure similar to what others have shown:
main_folder/
  data_source_X/
    data_source_X_model_1.sql
    ...
  data_source_Y/
    data_source_Y_model_1.sql
    ...

And we would like to have:
main_folder/
  data_source_X/
    model_1.sql
    ...
  data_source_Y/
    model_1.sql
    ...

But have a way to make the second approach be the same from dbt's "point of view".
We thought about something like (very rough sketch):
{% macro generate_model_name %}
  model["name"] = model["unique_id"].split(".")[-2:]
{% endmacro %}

There is one, generate_alias_name, but the model make doesn't change the underlying database table does - model names still need to be unique.
Annoying but I'm sort of used to it now, the unique model makes are easier to find in editors and fits nicely with the "how we structure our projects" guide on the dbt site, and if I really want my db to reflect different names then it's doable.
I'm keen on seeing this fixed properly but hasn't ended up being a blocker on us jumping over from dataform - lack of namespace support was a problem but aliases worked well enough.


test: pkg/envoy/lds listenerBuilder.getOutboundTCPFilterChainForService()

In pkg/envoy/lds listenerBuilder.getOutboundTCPFilterChainForService() does not have good unit test coverage.
It would be great to write a small test for this function.

ref #1489