Table of Contents

• The Problem with AI and AI Detectors
  
  • What Has the Chamber Learned?
    
  • Size Matters
    
• AI and the Brotherhood
  
  • Cite, Cite, Cite
    
  • Closing Thoughts
    
• Important Links
  

The Problem with AI and AI Detectors

TL:DR - AI is getting smarter and the Chamber of Justice asks you to engage us if you suspect AI has been utilized.

Two years ago, while running for Justicar, I brought forward Artificial Intelligence (AI) as the top area I was concerned about for the future of the Brotherhood. In short order myself and my staff saw two cases related to the inappropriate use of AI.

• Case 070 which involved a negative operator utilizing AI to create realistic new member joins and then test their ability to gain access to the site (e.g., via the Wiki) and make changes before a system administrator would notice.
  
• Case 071 which involved the utilization of AI to create fictional entries for competitions.
  

In addition to these two cases the Chamber has investigated several situations of potential AI utilization for graphic competitions, Shadow Academy exams, fiction competitions, and utilization in Role Playing.

What Has the Chamber Learned?

TL:DR - AI is getting smarter and AI detectors are struggling to keep up.

TL:DR - Just because AI was utilized doesn't mean AI created the whole submission.

During the investigation of Case 071 the Chamber was consistently finding results of >90% on fictional submissions for AI utilization. In most cases, the results were 100%.

These results were consistent across multiple AI detector tools, ZeroGPT, Quillbot, and Grammarly to name a few. Further, the way the text read had an unnatural quality to it.

As the sands of Tatooine whispered tales of forgotten glory and untold riches, <redacted> found herself inexorably drawn towards the promise of adventure that beckoned from beyond the horizon. And though the path ahead remained shrouded in uncertainty, she knew that with each step she took, she drew closer to the fulfillment of her resolution.

Phrases like "fulfillment of her resolution" stand out as phrasing most human writers would not utilize. Now, in comparison, is a recent prompt I entered into ChatGPT.

Kamjin the Hutt felt a rare flicker of excitement as his ship approached Nal Hutta, the humid glow of the planet stirring memories he hadn’t visited in decades. Despite the grime and the politics, there was something comforting—almost nostalgic—about returning to the heart of Hutt space, where every deal, every betrayal, felt like coming home.

For those familiar with my writing, or just reading this report, you'll notice that my utilization of commas are similar to those in the above extract but the use of em dashes is not. However, when running this threw an AI detector, in this case ZeroGPT. It returns as 0% AI detected (e.g., 100% human writing).

Now, if I run the whole report, as written up to this point, it returns 11.14% AI and still detects the portion of writing from Case 071 as being generated by AI.

AI is still imperfect...or, phrased differently, is to perfect to be human. One of the biggest tells in AI generated content is the use of em dashes versus en dashes. Here, I'll turn to Copilot (AI) to explain the differences.

• Em dash (—): This is the longer of the two. It’s used to indicate a break in thought, an interruption, or to add emphasis. Think of it as a more dramatic pause than a comma or parentheses.
  Example: He was looking for one thing—revenge.
  
• En dash (–): Shorter than an em dash, but longer than a hyphen, the en dash primarily indicates ranges (such as numbers or dates) and connections between things.
  Example: Pages 10–15 cover the basics.
  

For most of us, humans, we end up using en dashes because typing programs (e.g., gDoc, Word, etc.) will automatically insert them when we place a regular - in between two items. It is rare for these documents to correct it to an em dash, which is longer, into the document nor is it easy to remember how to create an em dash if you're writing from a keyboard (ALT+0151 or writing -- in Word).

Now, this isn't a silver bullet as mobile keyboards tend to have a pop-up if you hold the hyphen key.

Size Matters

TL:DR - The smaller the sample size, the higher the likelihood of a positive result in an AI detector.

Before you all giggle, I'm referencing the sample size of the text. AI Detectors work best when there is a large sample size, such as in the thousands of words range, to review for consistency in writing. The larger the sample size the less likely the writing will be technically correct for a human writer.

What this means is that short answers - such as those utilized for the Shadow Academy or in Role Playing sessions where Discord has a word cap - is more likely to pop as AI. While researching a recent RP for suspected AI utilization I ran my own recent RP writing through ZeroGPT and found that I was consistently registering 40-60% for AI.

Let's all accept into the record that I am the least likely to utilizing AI for cheating at writing generating Hutt images.

AI and the Brotherhood

TL:DR - AI detectors are a tool, but not the only tool, we have to detect cheating.

So, where does that leave the Brotherhood in regard to AI? Candidly, not that much different than where we were before AI. The best detection for cheating remains the humancentric approach of researching the individual, their writing over time, and their general behavior.

For those who have judged fiction competitions can attest, you can tell when someone's writing in-person doesn't match their fiction writing and, if they've been submitting to competitions multiple times, if their writing style changes dramatically.

Cite, Cite, Cite

TL:DR - If you're going to use AI to proof your writing, cite it.

AI is a wonderful tool and, just as we do with other tools, cite your sources. For most of us, who consistently write, we have individuals who proof our writing. I cannot thank Reiden enough for all the times over the years that he's read my writing and corrected punctuation, grammar, and highlighted phrases/sentences/blocks of text that didn't read well.

We do not require people to cite that they had someone proof read their writing. However, if you're utilizing AI to proof read your writing (e.g., Grammarly) and you choose to replace sentences with AI rewritten sentences it is a good practice to note that you proofed your writing with AI. Competition organizers, especially those who have worked with me over the last few years, tend to run all the submissions through AI detectors (though this is not required).

Closing Thoughts

TL:DR - AI is here to stay, so we adapt.

While the above review cites fiction as a primary driver of AI utilization and recommends that individuals cite when they utilize AI to proof their submissions we have, so far, not delved into AI for artistic programs. The same practice should be carried forward.

Cite the utilization.

If you are creating an image and you use AI to insert a background or to clean up ink lines, cite those elements in the submission.

The goal here is not to invalidate all submission that utilize AI nor to push them fully into the realm of participation only. In an age where AI detectors are imperfect we want to minimize the potential for individuals to be flagged for suspected cheating. AI is here to stay. It has already become integrated into our daily lives and nearly all the applications or devices that we are utilizing.

We want people to use these tools to improve their work but not to replace their work!

I'd simplify it in these two examples.

• Utilizing AI to proof your writing for wordiness, grammar, and readability (and cited) - Appropriate
  
• Utilizing AI to create a clan fiction for a news report, that is not submitted for clusters and is cited - Appropriate
  
• Utilizing AI to write the majority of a fiction being submitted for a competition or clusters - Inappropriate
  
• Utilizing AI to solve a puzzle, cypher, riddle, SA Exam, or other competition to get the correct answer in the shortest amount of time - Inappropriate
  

As I noted above regarding sample sizes, here at the conclusion of this report, it reads as <2% AI utilization.

Important Links

• The Covenant
• Dark Brotherhood Chat Policies
• Policies Category

It is my hope that this helps give you greater insight into how the Chamber of Justice addresses

Meanwhile, if you have any questions relating to Brotherhood policies and the Covenant, or you just want to chat about any concerns, feel free to reach out by email ([Log in to view e-mail addresses]) or message me on Discord.