My prompt induced infinite loops with QwQ

#35
by hanaoverride - opened

I tried prompt with MTG rules inference based on Comprehensive Rules(Rules Text that competitive MTG player mostly look up).

Original Prompt is exactly same as following one:

"Comprehensive Rules에 따라서 판단할때, 필드에 아무 지속물도 없고, 내 덱에 카드가 한장 있는 상태에서 Faithless Looting을 발동해. 그러면 카드를 버리는 것까지 효과를 완전히 처리한 후 패배해, 혹은 두번째 카드를 뽑으려고 시도하는 시점에서 패배해? 해당하는 CR의 넘버까지 붙여서 해설해줘"

(English Translation)
"According to the Comprehensive Rules, if I cast Faithless Looting when there are no permanents on the field and only one card in my library, do I lose after fully resolving the effect including discarding, or do I lose at the moment when attempting to draw the second card? Please explain with the relevant CR numbers."

QwQ initially followed a proper reasoning process, but while referencing the Comprehensive Rules, it fell into an infinite loop by repeatedly searching for the next item without end.
(EDIT : Although it referenced wrong text of Faithless Looting, I focused more on its reasoning process)

QwQ's original Response in brief:

--- Response Start ---
Faithless Looting는 다음과 같은 효과를 가지고 있습니다: "Draw two cards. Then, reveal the top card of your library. If it's a creature card, put it into your hand. Otherwise, put it into your graveyard." 이 카드는 당신의 덱의 위쪽 카드를 드러내고, 그 카드가 생물 카드이면 손으로 가져오고, 그렇지 않으면 무덤으로 보냅니다.

문제 상황은 다음과 같습니다: 필드에 아무 지속물도 없고, 당신의 덱에 카드가 한 장만 남아 있습니다. 이 상태에서 Faithless Looting을 발동합니다.

우선, Faithless Looting의 효과는 다음과 같이 진행됩니다:

  1. 두 장의 카드를 뽑습니다.

  2. 그 다음, 당신의 덱의 위쪽 카드를 드러냅니다.

  3. 만약 그 카드가 생물 카드이면, 손으로 가져옵니다.

  4. 그렇지 않으면, 무덤으로 보냅니다.

문제는 덱에만 한 장의 카드가 남아 있는 상황에서 이 효과를 처리할 때, 언제 패배가 발생하는지에 대한 것입니다.

먼저, Comprehensive Rules (CR)를 살펴보아야 합니다.

CR 104.3a: "If a player is required to draw more cards than are left in his or her library, he or she draws the remaining cards in his or her library and then shuffles his or her graveyard to form a new library. Then, that player draws the remaining number of cards he or she is required to draw."

그러나 이 규칙은 덱에 남은 카드 수가 뽑아야 할 카드 수보다 적을 때 적용됩니다. 현재 상황에서는 Faithless Looting이 두 장의 카드를 뽑도록 하며, 덱에 한 장의 카드만 있습니다.

따라서, 첫 번째로 두 장의 카드를 뽑는 단계에서 문제 발생이 예상됩니다.

CR 120.3: "Drawing a card that can't be drawn because there are no cards in the library results in no cards being drawn, unless another effect specifies what happens in that situation."

하지만 Faithless Looting는 뽑을 카드가 없을 때 특별한 효과를 명시하고 있지 않습니다.

CR 104.3b: "If a player is required to draw more cards than are left in his or her library, he or she draws the remaining cards in his or her library and then shuffles his or her graveyard to form a new library. Then, that player draws the remaining number of cards he or she is required to draw."

이 규칙에 따르면, 뽑아야 할 카드 수가 덱에 남은 카드 수보다 많을 경우, 덱에 남은 카드를 모두 뽑은 후, 무덤을 섞어서 새로운 덱을 만든 다음, 남은 수의 카드를 뽑습니다.

하지만 현재 상황에서, 덱에 한 장의 카드만 있습니다. 따라서, 두 장의 카드를 뽑으려고 하면, 한 장을 뽑고 나서 무덤이 비어있을 것입니다. 만약 무덤이 비어 있다면, CR 104.3c를 참조해야 합니다.

CR 104.3c: "If a player is required to draw a card but has no cards in his or her library and no cards in his or her graveyard, that player doesn't draw a card."

그러나 이 경우에는, 덱에 한 장의 카드가 있고, 무덤은 비어 있는 상태입니다. 따라서, 첫 번째 카드를 뽑으면 덱이 비어 있고, 무덤은 여전히 비어 있습니다. 이때, 두 번째 카드를 뽑으려고 하면, 덱이 비어 있어 뽑을 카드가 없습니다. 따라서, CR 104.3c에 따라 두 번째 카드는 뽑을 수 없습니다.

그러나 Faithless Looting는 "Draw two cards."라고 명시되어 있으므로, 두 장의 카드를 뽑아야 합니다. 그러나 덱에만 한 장의 카드가 있기 때문에, 한 장을 뽑고 나서는 뽑을 수 없습니다.

이 상황에서, Comprehensive Rules를 찾아보니, CR 119.3a: "To draw a card, a player removes the top card of his or her library from the library and puts it into his or her hand. This is done as a single action."

그리고 CR 119.3b: "If a player is instructed to draw more than one card, he or she performs this action that many times."

따라서, 두 장의 카드를 뽑는 것은 뽑는 동작을 두 번 수행하는 것입니다.

첫 번째 뽑기: 덱에 한 장의 카드가 있으므로, 그 카드를 뽑습니다.

두 번째 뽑기: 덱이 비어 있으므로, CR 104.3c에 따라 뽑을 카드가 없습니다.

그러나, 이때 패배는 발생하지 않습니다. 패배는 특별한 조건 하에서만 발생합니다.

CR 104.1: "If a player is required to draw more cards than are left in his or her library, he or she draws the remaining cards, then loses the game the next time a player would receive priority. (This is a state-based action. See rule 704.)"

즉, 덱에서 더 이상 뽑을 카드가 없을 때, 그 다음으로 우선권을 받을 때 패배합니다.

그러나, Faithless Looting의 경우, 두 장의 카드를 뽑는 동작을 수행하려고 합니다. 첫 번째 카드는 뽑을 수 있지만, 두 번째 카드를 뽑을 수 없습니다.

이때, 패배는 언제 발생하는지가 문제입니다.

CR 104.3a: "If a player is required to draw more cards than are left in his or her library, he or she draws the remaining cards in his or her library and then shuffles his or her graveyard to form a new library. Then, that player draws the remaining number of cards he or she is required to draw."

하지만, 현재 상황에서는 무덤이 비어 있습니다. 따라서, CR 104.3c에 따라 두 번째 카드는 뽑을 수 없습니다.

그러나, CR 104.1에 따르면, 뽑을 카드가 없을 때, 그 다음으로 우선권을 받을 때 패배합니다.

따라서, Faithless Looting의 효과를 처리하는 동안, 두 번째 카드를 뽑을 수 없을 때, 그 다음으로 우선권을 받을 때 패배하게 됩니다.

하지만, Faithless Looting의 효과는 계속 진행됩니다.

다음 단계는 "Then, reveal the top card of your library."입니다.

그러나, 덱이 이미 비어 있으므로, 드러낼 카드가 없습니다.

CR 104.4: "If an effect attempts to have a player reveal cards from his or her library, and there are no cards in that library, no cards are revealed. The effect doesn't fail unless specifically indicated otherwise."

따라서, 덱에 카드가 없으므로, 드러낼 카드가 없습니다. 이 경우, 효과는 실패하지 않습니다.

그러나, Faithless Looting의 효과는 "Then, reveal the top card of your library. If it's a creature card, put it into your hand. Otherwise, put it into your graveyard."입니다.

만약 드러낼 카드가 없으면, "it's a creature card" 조건을 평가할 수 없습니다.

이 경우, Comprehensive Rules를 살펴보아야 합니다.

CR 608.2b: "Some activated abilities and some triggered abilities include cost payments and/or the word ‘pay.’ These abilities are functions that a player can choose to pay for with the listed costs in order to have them do something."

하지만 이 경우는 해당하지 않습니다.

CR 608.2c: "The value of an activated ability on the stack is determined when it's activated."

아니오, 이 경우도 아닙니다.

CR 608.2g: "Some continuous effects are replacement effects or prevention effects that apply to objects or events that may exist or occur in the future. These effects wait for a particular event to happen; when it does, the effect replaces or prevents it."

아니오.

CR 608.2h: "Some continuous effects are state-based effects. These effects are written as conditions (using the word "if") followed by effects (using the word "then")."

아니오.

CR 608.2i: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

CR 608.2j: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

CR 608.2k: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

CR 608.2l: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

CR 608.2m: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

CR 608.2n: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

CR 608.2o: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

CR 608.2p: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

CR 608.2q: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

CR 608.2r: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

CR 608.2s: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

CR 608.2t: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

CR 608.2u: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

CR 608.2v: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

CR 608.2w: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

CR 608.2x: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

CR 608.2y: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

CR 608.2z: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

CR 608.2aa: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

CR 608.2ab: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

CR 608.2ac: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

CR 608.2ad: "Some continuous effects are called delayed triggers. These effects are written as abilities that would normally be triggered abilities, but are worded as continuous effects."

아니오.

(Continuous CR References)

--- Response End ---
(Loop stopped around printing CR 608.2md)
This issue may cause significant disruption in user experience, especially for tasks requiring complex reasoning over structured data like game rules. Addressing this problem would improve the model's reliability and usability.

In my opinion, the direct instruction to reference the Comprehensive Rules likely induced the infinite loop. While I cannot confirm this with certainty, I would like to discuss this with others in the community.

EDIT :
I think This infinite loop issue likely arises due to the combination of the following factors:

  1. MTG Comprehensive Rules’ Nested Structure
    The MTG Comprehensive Rules are highly interconnected, with many rules referencing others to resolve edge cases.
    This recursive nature might have caused QwQ to continue searching for the "next relevant rule" indefinitely,
    failing to establish an endpoint for its reasoning process.

  2. Incorrect Faithless Looting Text Reference
    QwQ's reasoning process was initially correct, but its reference to Faithless Looting’s effect text was inaccurate.
    (It's actual oracle text is "draw two cards, then discard two cards.")
    This discrepancy might have led QwQ to base its reasoning on a flawed premise, further complicating its ability to navigate the rules effectively.

  3. Lack of a Clear Stopping Condition
    QwQ appears to lack a mechanism to identify when it has exhausted all meaningful references in the Comprehensive Rules.
    Instead, it continued attempting to reference new rules, which ultimately resulted in repetitive outputs like "CR 608.2x: No," "CR 608.2y: No," etc., leading to an infinite loop.

  4. Prompt Ambiguity and Model Overinterpretation
    The phrasing of my prompt (“Please explain with the relevant CR numbers”) may have inadvertently encouraged QwQ to reference every possible related rule without a termination criterion.
    This suggests that the model might need better handling of broad or open-ended tasks when dealing with structured, highly recursive datasets like the Comprehensive Rules.
    While I understand this issue is a phenomenon caused by LLM's probabilistic decision-making, general users tend to use it without considering such aspects.

※ Suggested Improvements
I think adding this features would reduce such a flaw:

(1) Implement Stop Conditions: Ensure the model can identify when it has fully addressed a query, particularly when working with recursive or interconnected data sources.
(2) Validation : Improve the ability to verify and cross-check referenced data, such as card texts or external rulebooks, before proceeding with in-depth reasoning.
(3) Enhance Recursive Query Handling: Introduce safeguards for scenarios where extensive cross-referencing might result in infinite loops.

MTG's Comprehensive Rules are indeed similar to real-world legal codes,
and the process of referencing and interpreting them bears similarities to legal interpretation.
If this issue can be improved, it could help reduce flaws in legal applications as well!

I faced something similar when i was trying to fine tune the model on Long COT and its kept going in loops when asked anything

Sign up or log in to comment