Congratulations!

[Valid RSS] This is a valid RSS feed.

Recommendations

This feed is valid, but interoperability with the widest range of feed readers could be improved by implementing the following recommendations.

Source: http://feeds.feedburner.com/WaldekMastykarz

  1. <?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:webfeeds="http://webfeeds.org/rss/1.0"><channel><title>Waldek Mastykarz</title><description>Innovation matters</description><link>https://blog.mastykarz.nl/</link><image><url>https://blog.mastykarz.nl/favicon.ico</url><title>Waldek Mastykarz</title><link>https://blog.mastykarz.nl/</link></image><atom:link href="https://blog.mastykarz.nl/feed.xml" rel="self" type="application/rss+xml"/><pubDate>Thu, 19 Jun 2025 10:39:00 GMT</pubDate><lastBuildDate>Thu, 19 Jun 2025 10:39:00 GMT</lastBuildDate><webfeeds:analytics id="UA-3652888-1" engine="GoogleAnalytics"/><ttl>60</ttl><item><title>Benchmark models using OpenAI-compatible APIs</title><link>https://blog.mastykarz.nl/benchmark-models-openai-compatible-apis/</link><guid isPermaLink="true">https://blog.mastykarz.nl/benchmark-models-openai-compatible-apis/</guid><description>&lt;p&gt;&lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2025/06/banner-language-model-results.png&quot; alt=&quot;Benchmark models using OpenAI-compatible APIs&quot; class=&quot;webfeedsFeaturedVisual&quot; /&gt;&lt;/p&gt;&lt;p&gt;Recently, I wrote about &lt;a href=&quot;/language-model-benchmarks-story/&quot;&gt;why you should write your own benchmarks for language models&lt;/a&gt; to see how they work for your app. I also shared a ready-to-use &lt;a href=&quot;https://github.com/waldekmastykarz/ollama-compare&quot;&gt;Jupyter Notebook that allows you to evaluate language models on Ollama&lt;/a&gt;. I&apos;ve just published a new version of the &lt;a href=&quot;https://github.com/waldekmastykarz/openai-compare&quot;&gt;notebook which now supports any language model host that exposes OpenAI-compatible APIs&lt;/a&gt;.&lt;/p&gt;
  2. &lt;p&gt;Like the previous version, the new notebook shows you how well your select language models perform for each scenario, and overall.&lt;/p&gt;
  3. &lt;p&gt;&lt;picture&gt;
  4.  &lt;source srcset=&quot;https://blog.mastykarz.nl/assets/images/2025/06/language-model-test-oai-per-scenario.webp&quot; type=&quot;image/webp&quot;&gt;
  5.  &lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2025/06/language-model-test-oai-per-scenario.png&quot; alt=&quot;Bar chart showing benchmark results for different language models across four tasks: getCalendarForUser, postCalendarForUser, getUser, and postCalendarForUser. The chart displays average combined scores on the y-axis from 0 to 1.0, with a horizontal line at 0.8 indicating the threshold. Six models are compared: Phi-3.5-mini-instruct-generic-gpu, Phi-4-mini-instruct-generic-gpu, mistralai-Mistral-7B-Instruct-v0-2-generic-gpu, qwen2.5-7b-instruct-generic-gpu, qwen2.5-1.5b-instruct-generic-gpu, and qwen2.5-0.5b-instruct-generic-gpu. Most models achieve perfect scores of 1.0 across tasks, with some exceptions showing lower scores particularly for the smallest qwen2.5-0.5b model. The chart title reads Dev Proxy v0.29.0 OpenAPI Operation ID and includes a legend on the right side identifying each model with different colored bars.&quot;&gt;
  6. &lt;/picture&gt;&lt;/p&gt;
  7. &lt;p&gt;&lt;picture&gt;
  8.  &lt;source srcset=&quot;https://blog.mastykarz.nl/assets/images/2025/06/language-model-test-oai-overall.webp&quot; type=&quot;image/webp&quot;&gt;
  9.  &lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2025/06/language-model-test-oai-overall.png&quot; alt=&quot;Bar chart displaying benchmark results for six language models tested across four OpenAI API operations. The vertical axis shows Average Combined Score from 0 to 1.0, with a red horizontal threshold line at 0.8. The models tested are Phi-3.5-mini-instruct-generic-gpu achieving a perfect score of 1.0, Phi-4-mini-instruct-generic-gpu scoring 0.999, qwen2.5-7b-instruct-generic-gpu scoring 0.975, qwen2.5-1.5b-instruct-generic-gpu scoring 0.849, mistralai-Mistral-7B-Instruct-v0-2-generic-gpu scoring 0.763, and qwen2.5-0.5b-instruct-generic-gpu scoring 0.556. All bars are colored in blue against a white background with a grid. The chart title reads Dev Proxy v0.29.0 OpenAPI Operation ID. The visualization demonstrates that larger models generally perform better, with only the two smallest models falling below the 0.8 performance threshold. The professional appearance suggests this is from a technical benchmarking study or research publication.&quot;&gt;
  10. &lt;/picture&gt;&lt;/p&gt;
  11. &lt;p&gt;What&apos;s changed, is that you can now test models running not just on Ollama but on any host that exposes OpenAI-compatible APIs, be it &lt;a href=&quot;https://learn.microsoft.com/azure/ai-foundry/foundry-local/what-is-foundry-local&quot;&gt;Foundry Local&lt;/a&gt;, &lt;a href=&quot;https://ollama.com/&quot;&gt;Ollama&lt;/a&gt;, &lt;a href=&quot;https://learn.microsoft.com/azure/ai-foundry/what-is-azure-ai-foundry&quot;&gt;Azure AI Foundry&lt;/a&gt;, or &lt;a href=&quot;https://openai.com/&quot;&gt;OpenAI&lt;/a&gt;. When selecting the models to test, you can now specify the language model host, and an API key if you need one:&lt;/p&gt;
  12. &lt;pre&gt;&lt;code class=&quot;language-python&quot;&gt;# Foundry Local
  13. client = OpenAI(
  14.    api_key=&amp;quot;local&amp;quot;,
  15.    base_url=&amp;quot;http://localhost:5272/v1&amp;quot;
  16. )
  17. models = [&amp;quot;qwen2.5-0.5b-instruct-generic-gpu&amp;quot;, &amp;quot;qwen2.5-1.5b-instruct-generic-gpu&amp;quot;, &amp;quot;qwen2.5-7b-instruct-generic-gpu&amp;quot;, &amp;quot;Phi-4-mini-instruct-generic-gpu&amp;quot;, &amp;quot;Phi-3.5-mini-instruct-generic-gpu&amp;quot;, &amp;quot;mistralai-Mistral-7B-Instruct-v0-2-generic-gpu&amp;quot;]
  18. &lt;/code&gt;&lt;/pre&gt;
  19. &lt;p&gt;By default, the notebook includes a configuration for Foundry Local and Ollama.&lt;/p&gt;
  20. &lt;p&gt;Another thing that I&apos;ve changed is storing prompts in &lt;a href=&quot;https://prompty.ai&quot;&gt;Prompty&lt;/a&gt; files. Using Prompty makes it convenient to separate the prompts from the notebook&apos;s code. It&apos;s also easy for you to quickly run a prompt and see the result of any changes you could make. I suggest you try it for yourself to see how easy it is.&lt;/p&gt;
  21. &lt;p&gt;If you&apos;re using language models in your app, you should be testing them to choose the one that&apos;s best for your scenario. &lt;a href=&quot;https://github.com/waldekmastykarz/openai-compare&quot;&gt;This Jupyter Notebook&lt;/a&gt; helps you do that.&lt;/p&gt;
  22. </description><pubDate>Thu, 19 Jun 2025 10:39:00 GMT</pubDate></item><item><title>Language model benchmarks only tell half a story</title><link>https://blog.mastykarz.nl/language-model-benchmarks-story/</link><guid isPermaLink="true">https://blog.mastykarz.nl/language-model-benchmarks-story/</guid><description>&lt;p&gt;&lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2025/06/banner-language-model-results.png&quot; alt=&quot;Language model benchmarks only tell half a story&quot; class=&quot;webfeedsFeaturedVisual&quot; /&gt;&lt;/p&gt;&lt;p&gt;When it comes to language models, we tend to look at benchmarks to decide which model is the best to use in our application. But benchmarks only tell half a story. Unless you&apos;re building an all-purpose chat application, what you should be actually looking at is how well a model works for your application.&lt;/p&gt;
  23. &lt;p&gt;&lt;em&gt;This article is based on a benchmark I put together to evaluate which of the language models available on Ollama is best suited for Dev Proxy. You can find the &lt;a href=&quot;https://github.com/waldekmastykarz/ollama-compare&quot;&gt;working prototype&lt;/a&gt; on GitHub. I&apos;m working on a similar solution that supports OpenAI-compatible APIs and will publish it shortly.&lt;/em&gt;&lt;/p&gt;
  24. &lt;h2&gt;When the best isn&apos;t the best&lt;/h2&gt;
  25. &lt;p&gt;When we started looking at integrating local language models with &lt;a href=&quot;https://aka.ms/devproxy&quot;&gt;Dev Proxy&lt;/a&gt;, to improve generating OpenAPI specs, we used Phi-3 on Ollama. It seemed like a reasonable choice: the model was pretty quick and capable, at least according to benchmarks. It turned out, that we couldn&apos;t have picked a worse model for our needs.&lt;/p&gt;
  26. &lt;p&gt;&lt;picture&gt;
  27.  &lt;source srcset=&quot;https://blog.mastykarz.nl/assets/images/2025/06/language-model-results-per-model.webp&quot; type=&quot;image/webp&quot;&gt;
  28.  &lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2025/06/language-model-results-per-model.png&quot; alt=&quot;Bar chart showing average combined scores for four language models in optimized operation ID generation. From left to right: phi3.5 scored 0.967, llama3.2 scored 0.954, qwen2.5:3b scored 0.922, and phi3 scored 0.517. A red horizontal line at 0.8 indicates the performance threshold. The chart demonstrates phi3.5 and llama3.2 as top performers, while phi3 significantly underperforms compared to the others.&quot;&gt;
  29. &lt;/picture&gt;&lt;/p&gt;
  30. &lt;h2&gt;The problem with language model benchmarks&lt;/h2&gt;
  31. &lt;p&gt;We love benchmarks and comparisons. In the world of abundance, they offer an easy answer: which product is the best and which one should we choose. The thing is though, that a benchmark is based on &lt;em&gt;some&lt;/em&gt; criteria. And unless you plan to use the product exactly based on these criteria, the benchmark is, well, useless. Sure, the results tell you something about the product, but it&apos;s hardly the right information for you to choose.&lt;/p&gt;
  32. &lt;p&gt;So rather than relying on standard benchmarks, when it comes to choosing the best model for your application, what you actually need is to tell how well a model works for &lt;strong&gt;your scenario&lt;/strong&gt;. You need &lt;strong&gt;your own benchmark&lt;/strong&gt;.&lt;/p&gt;
  33. &lt;h2&gt;Build your own language model benchmark&lt;/h2&gt;
  34. &lt;p&gt;Evaluating language models is a science. There are many papers written on the subject, and if you&apos;re building a language model for general use, you should definitely read them. But if you&apos;re reading this article, probably all you care about is which model you should use in your application. You don&apos;t care about most cases. You care about &lt;strong&gt;your case&lt;/strong&gt;, and how the model works &lt;strong&gt;for you&lt;/strong&gt;. To understand how well a model works for you, you should put together a benchmark.&lt;/p&gt;
  35. &lt;h3&gt;Benchmark building blocks&lt;/h3&gt;
  36. &lt;p&gt;Typically, a benchmark consists of the following elements:&lt;/p&gt;
  37. &lt;ul&gt;
  38. &lt;li&gt;one or more test cases including the test scenario (prompt and parameters) and expected outcome&lt;/li&gt;
  39. &lt;li&gt;evaluation criteria&lt;/li&gt;
  40. &lt;li&gt;scoring system&lt;/li&gt;
  41. &lt;/ul&gt;
  42. &lt;p&gt;Test cases represent how your application is using the language model. For each test case, you need to include one or more expected results. Evaluation criteria is how you&apos;re going to verify in a repeatable way, that the actual result for a test case is like the one you expected. The scoring system allows you to quickly compare the results for each model you tested and tell which one works the best for you.&lt;/p&gt;
  43. &lt;h3&gt;Benchmark for a language model&lt;/h3&gt;
  44. &lt;p&gt;Since we&apos;re talking about testing language models, there are a few more things that you need to consider while building a benchmark.&lt;/p&gt;
  45. &lt;p&gt;Language models are generative and their output is non-deterministic. If you run the same prompt twice, you&apos;ll likely get two different answers. This means, that:&lt;/p&gt;
  46. &lt;ol&gt;
  47. &lt;li&gt;To get a representative result, you need to run each test case several times.&lt;/li&gt;
  48. &lt;li&gt;There are probably several correct answers to each test case.&lt;/li&gt;
  49. &lt;li&gt;You can&apos;t use a simple string comparison to evaluate the actual and expected output.&lt;/li&gt;
  50. &lt;/ol&gt;
  51. &lt;h4&gt;Evaluating language model output&lt;/h4&gt;
  52. &lt;p&gt;To solve the first challenge, you need to run each test case and get an average score.&lt;/p&gt;
  53. &lt;p&gt;For the second part, you should provide several reference examples. When scoring language model&apos;s output against these references, take the highest score.&lt;/p&gt;
  54. &lt;p&gt;The final problem is a bit trickier. The good news is, that it&apos;s not a new problem, and there are several commonly used functions for scoring language model output, such as BLEU, ROUGE, BERT or edit distance. The challenging part is, that you&apos;ll likely need a combination of these functions to determine the efficacy of a model for your scenario. To do that, you&apos;ll define a weighted score, where each function gets assigned importance (weight) for the overall score.&lt;/p&gt;
  55. &lt;h4&gt;Example evaluations for Dev Proxy scenarios&lt;/h4&gt;
  56. &lt;p&gt;To put it all in practice, let&apos;s look at two scenarios that we use in Dev Proxy.&lt;/p&gt;
  57. &lt;h5&gt;Generate API operation ID&lt;/h5&gt;
  58. &lt;p&gt;One way we use language models in Dev Proxy is to, given a request method and URL, generate the ID of the API operation when generating OpenAPI specs. For example, for a request like: &lt;code&gt;POST https://graph.microsoft.com/users/{users-id}/calendars&lt;/code&gt; we want to get &lt;code&gt;addUserCalendar&lt;/code&gt; or an equivalent.&lt;/p&gt;
  59. &lt;p&gt;For this case, we use the following functions and weights:&lt;/p&gt;
  60. &lt;table&gt;
  61. &lt;thead&gt;
  62. &lt;tr&gt;
  63. &lt;th&gt;Function&lt;/th&gt;
  64. &lt;th style=&quot;text-align:right&quot;&gt;Weight&lt;/th&gt;
  65. &lt;/tr&gt;
  66. &lt;/thead&gt;
  67. &lt;tbody&gt;
  68. &lt;tr&gt;
  69. &lt;td&gt;BERT-F&lt;/td&gt;
  70. &lt;td style=&quot;text-align:right&quot;&gt;0.45&lt;/td&gt;
  71. &lt;/tr&gt;
  72. &lt;tr&gt;
  73. &lt;td&gt;Edit distance&lt;/td&gt;
  74. &lt;td style=&quot;text-align:right&quot;&gt;0.10&lt;/td&gt;
  75. &lt;/tr&gt;
  76. &lt;tr&gt;
  77. &lt;td&gt;ROUGE-1&lt;/td&gt;
  78. &lt;td style=&quot;text-align:right&quot;&gt;0.25&lt;/td&gt;
  79. &lt;/tr&gt;
  80. &lt;tr&gt;
  81. &lt;td&gt;ROUGE-L&lt;/td&gt;
  82. &lt;td style=&quot;text-align:right&quot;&gt;0.20&lt;/td&gt;
  83. &lt;/tr&gt;
  84. &lt;/tbody&gt;
  85. &lt;/table&gt;
  86. &lt;p&gt;We assign the most weight to BERT-F because we want the generated ID to be semantically accurate. It would be nice if the generated ID matched one of our expected IDs (ROUGE-L), and the words overlap the better (ROUGE-1). Finally, we don&apos;t want the ID to deviate too much from our examples (Edit distance).&lt;/p&gt;
  87. &lt;p&gt;You could have endless debate about each weight. It doesn&apos;t really matter if you assign 0.45 or 0.43 to a function. What matters, is that you can explain what characteristics of the answer you care about and why, and that answers that are good in your eyes, get good scores.&lt;/p&gt;
  88. &lt;h5&gt;Generate API operation description&lt;/h5&gt;
  89. &lt;p&gt;Another scenario for which we use language models in Dev Proxy is to generate an operation&apos;s description for use in an OpenAPI spec. For example, given a request &lt;code&gt;GET https://api.contoso.com/users/{users-id}/calendars&lt;/code&gt;, we want to get &lt;code&gt;Retrieve a user&apos;s calendars&lt;/code&gt; or something similar. For this case, we use the following evaluation criteria:&lt;/p&gt;
  90. &lt;table&gt;
  91. &lt;thead&gt;
  92. &lt;tr&gt;
  93. &lt;th&gt;Function&lt;/th&gt;
  94. &lt;th style=&quot;text-align:right&quot;&gt;Weight&lt;/th&gt;
  95. &lt;/tr&gt;
  96. &lt;/thead&gt;
  97. &lt;tbody&gt;
  98. &lt;tr&gt;
  99. &lt;td&gt;BERT-F&lt;/td&gt;
  100. &lt;td style=&quot;text-align:right&quot;&gt;0.45&lt;/td&gt;
  101. &lt;/tr&gt;
  102. &lt;tr&gt;
  103. &lt;td&gt;Edit distance&lt;/td&gt;
  104. &lt;td style=&quot;text-align:right&quot;&gt;0.10&lt;/td&gt;
  105. &lt;/tr&gt;
  106. &lt;tr&gt;
  107. &lt;td&gt;ROUGE-2&lt;/td&gt;
  108. &lt;td style=&quot;text-align:right&quot;&gt;0.25&lt;/td&gt;
  109. &lt;/tr&gt;
  110. &lt;tr&gt;
  111. &lt;td&gt;ROUGE-L&lt;/td&gt;
  112. &lt;td style=&quot;text-align:right&quot;&gt;0.20&lt;/td&gt;
  113. &lt;/tr&gt;
  114. &lt;/tbody&gt;
  115. &lt;/table&gt;
  116. &lt;p&gt;In this case, we replaced ROUGE-1 with ROUGE-2. When generating operation ID, we looked for overlap of the different tokes (eg. &lt;code&gt;getUserById&lt;/code&gt; vs. &lt;code&gt;getUserByIdentifier&lt;/code&gt; matches 3 out of 4 words). Since description is more elaborate, we decided to look at the matching word pairs than single words.&lt;/p&gt;
  117. &lt;p&gt;Again, this isn&apos;t &lt;em&gt;the&lt;/em&gt; way to evaluate a language model output. This is the way &lt;em&gt;we&lt;/em&gt; chose to use for &lt;em&gt;our&lt;/em&gt; scenario. I encourage you to experiment with different functions and weights to see what&apos;s working for you.&lt;/p&gt;
  118. &lt;h4&gt;A word on BERTScore&lt;/h4&gt;
  119. &lt;p&gt;BERTScore is a great and convenient way to see if the generated text is semantically similar to the reference. The only trouble is that, comparing to other scoring functions, weak results in BERTScore are scored up to 0.8 out of 1. So if you combine it with other functions, you&apos;ll get skewed results. To avoid this, you need to normalize its value, for example: x &amp;gt; 0.95 give full contribution (1), 0.75&amp;lt;= x &amp;lt;0.95, scale linearly, and for x &amp;lt; 0.75 drop to 0.&lt;/p&gt;
  120. &lt;h3&gt;Putting it all together&lt;/h3&gt;
  121. &lt;p&gt;Putting it all together, you&apos;ll end up with a solution that consists of the following building blocks:&lt;/p&gt;
  122. &lt;ul&gt;
  123. &lt;li&gt;test scenario based on a parametrized language model prompt&lt;/li&gt;
  124. &lt;li&gt;one or more test cases, each with values for the prompt parameters and several reference values&lt;/li&gt;
  125. &lt;li&gt;weighted score based on several scoring functions&lt;/li&gt;
  126. &lt;li&gt;test runner that:
  127. &lt;ul&gt;
  128. &lt;li&gt;runs the test scenario against a language model,&lt;/li&gt;
  129. &lt;li&gt;evaluates, and scores its output&lt;/li&gt;
  130. &lt;li&gt;presents the comparison result&lt;/li&gt;
  131. &lt;/ul&gt;
  132. &lt;/li&gt;
  133. &lt;/ul&gt;
  134. &lt;p&gt;&lt;picture&gt;
  135.  &lt;source srcset=&quot;https://blog.mastykarz.nl/assets/images/2025/06/language-model-results-per-model-per-case.webp&quot; type=&quot;image/webp&quot;&gt;
  136.  &lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2025/06/language-model-results-per-model-per-case.png&quot; alt=&quot;Bar chart comparing language model performance across four different tasks. Shows average combined scores for qwen2.5:3b, phi3.5, llama3.2, and phi3 models on getCalendarForUser, getCalendarForUser, getUser, and postCalendarForUser tasks. A horizontal red line at 0.8 marks the performance threshold. Most models score above 0.9 on the first three tasks, with phi3.5, llama3.2, and qwen2.5:3b performing consistently well. However, phi3 shows dramatically lower performance on the first task at approximately 0.146, while all other scores remain above 0.8. The chart demonstrates significant performance variation between models and tasks, with phi3 being particularly unsuitable for certain operations despite adequate performance on others.&quot;&gt;
  137. &lt;/picture&gt;&lt;/p&gt;
  138. &lt;p&gt;&lt;picture&gt;
  139.  &lt;source srcset=&quot;https://blog.mastykarz.nl/assets/images/2025/06/language-model-results-per-model.webp&quot; type=&quot;image/webp&quot;&gt;
  140.  &lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2025/06/language-model-results-per-model.png&quot; alt=&quot;Bar chart showing average combined scores for four language models in optimized operation ID generation. From left to right: phi3.5 scored 0.967, llama3.2 scored 0.954, qwen2.5:3b scored 0.922, and phi3 scored 0.517. A red horizontal line at 0.8 indicates the performance threshold. The chart demonstrates phi3.5 and llama3.2 as top performers, while phi3 significantly underperforms compared to the others.&quot;&gt;
  141. &lt;/picture&gt;&lt;/p&gt;
  142. &lt;p&gt;Check out the &lt;a href=&quot;https://github.com/waldekmastykarz/ollama-compare&quot;&gt;sample testing workbench&lt;/a&gt; that I created and that allows you to evaluate language models running on Ollama. I chose to build it as a Jupyter Notebook using Python, for several reasons:&lt;/p&gt;
  143. &lt;ul&gt;
  144. &lt;li&gt;Python has ready to use libraries with scoring functions, data manipulation and presentation logic&lt;/li&gt;
  145. &lt;li&gt;Since it&apos;s a fully fledged programming language, I can implement caching the language models and responses for quicker execution&lt;/li&gt;
  146. &lt;li&gt;Jupyter Notebooks:
  147. &lt;ul&gt;
  148. &lt;li&gt;allow you to combine code and content with additional information&lt;/li&gt;
  149. &lt;li&gt;can be run interactively&lt;/li&gt;
  150. &lt;li&gt;combined with VSCode extensions such as &lt;a href=&quot;https://marketplace.visualstudio.com/items?itemName=ms-toolsai.datawrangler&quot;&gt;Data Wrangler&lt;/a&gt; allow you to explore the variables in your notebook which is invaluable for debugging and understanding the results&lt;/li&gt;
  151. &lt;li&gt;persist last execution state. If you change something, you can conveniently re-run the notebook from the change part, speeding up the execution&lt;/li&gt;
  152. &lt;/ul&gt;
  153. &lt;/li&gt;
  154. &lt;/ul&gt;
  155. &lt;p&gt;I&apos;m working on a similar version based on OpenAI-compatible APIs, and which will also support &lt;a href=&quot;https://prompty.ai&quot;&gt;Prompty&lt;/a&gt; files.&lt;/p&gt;
  156. &lt;h2&gt;Trust but verify&lt;/h2&gt;
  157. &lt;p&gt;Next time you&apos;re building an application that uses a language model, be sure to test a few models and compare the results. Don&apos;t just blindly trust generic benchmarks that aren&apos;t representative of how you use language models in your applications, but see what&apos;s working best for you. When you change the model you use or update the prompt, re-run your tests and compare the results. You&apos;ll be surprised to see how much difference a model or a prompt can make. When you start testing how you use language models, not only you&apos;ll see which model works best for you, but you&apos;ll also discover which prompts you could improve. If a model scores well in one test, but poorly in another, and the scenarios aren&apos;t wildly different, try changing the prompt. Stay curious.&lt;/p&gt;
  158. </description><pubDate>Tue, 17 Jun 2025 15:11:40 GMT</pubDate></item><item><title>API request encoding matters</title><link>https://blog.mastykarz.nl/api-request-encoding-matters/</link><guid isPermaLink="true">https://blog.mastykarz.nl/api-request-encoding-matters/</guid><description>&lt;p&gt;&lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2025/06/banner-api-encoding.png&quot; alt=&quot;API request encoding matters&quot; class=&quot;webfeedsFeaturedVisual&quot; /&gt;&lt;/p&gt;&lt;p&gt;When you submit data to an API, the character encoding is crucial to ensure that the data is processed properly, especially if you&apos;re working with UTF characters.&lt;/p&gt;
  159. &lt;p&gt;Consider the following HTTP request:&lt;/p&gt;
  160. &lt;pre&gt;&lt;code class=&quot;language-http&quot;&gt;POST http://api.ecs.eu/feedback
  161. Content-Type: application/json
  162.  
  163. {
  164.  &amp;quot;id&amp;quot;: {{$randomInt 9 1000000}},
  165.  &amp;quot;feedback&amp;quot;: &amp;quot;🫠&amp;quot;,
  166.  &amp;quot;date&amp;quot;: &amp;quot;{{$datetime iso8601}}&amp;quot;
  167. }
  168. &lt;/code&gt;&lt;/pre&gt;
  169. &lt;p&gt;Notice, that the &lt;code&gt;content-type&lt;/code&gt; is set to &lt;code&gt;application/json&lt;/code&gt; without a specific encoding, and the &lt;code&gt;feedback&lt;/code&gt; property contains an emoji. Most likely, the API will respond with something like:&lt;/p&gt;
  170. &lt;pre&gt;&lt;code class=&quot;language-http&quot;&gt;HTTP/1.1 201 Created
  171. content-type: application/json; charset=utf-8
  172. Content-Length: 82
  173.  
  174. {
  175.  &amp;quot;id&amp;quot;: 972680,
  176.  &amp;quot;feedback&amp;quot;: &amp;quot;ð\x9f« &amp;quot;,
  177.  &amp;quot;date&amp;quot;: &amp;quot;2025-06-16T11:33:28.326Z&amp;quot;
  178. }
  179. &lt;/code&gt;&lt;/pre&gt;
  180. &lt;p&gt;Notice how the contents of the &lt;code&gt;feedback&lt;/code&gt; property are broken. This is caused by the lack of encoding in the request. To fix this, extend the &lt;code&gt;content-type&lt;/code&gt; header with the encoding:&lt;/p&gt;
  181. &lt;pre&gt;&lt;code class=&quot;language-http&quot;&gt;POST http://api.ecs.eu/feedback
  182. Content-Type: application/json; charset=utf-8
  183.  
  184. {
  185.  &amp;quot;id&amp;quot;: {{$randomInt 9 1000000}},
  186.  &amp;quot;feedback&amp;quot;: &amp;quot;😎&amp;quot;,
  187.  &amp;quot;date&amp;quot;: &amp;quot;{{$datetime iso8601}}&amp;quot;
  188. }
  189. &lt;/code&gt;&lt;/pre&gt;
  190. &lt;p&gt;Now, the API will respond with the correct encoding:&lt;/p&gt;
  191. &lt;pre&gt;&lt;code class=&quot;language-http&quot;&gt;HTTP/1.1 201 Created
  192. content-type: application/json; charset=utf-8
  193. Content-Length: 82
  194.  
  195. {
  196.  &amp;quot;id&amp;quot;: 972680,
  197.  &amp;quot;feedback&amp;quot;: &amp;quot;😎&amp;quot;,
  198.  &amp;quot;date&amp;quot;: &amp;quot;2025-06-16T11:33:28.326Z&amp;quot;
  199. }
  200. &lt;/code&gt;&lt;/pre&gt;
  201. &lt;p&gt;This issue caught me off guard while presenting how to use &lt;a href=&quot;https://aka.ms/devproxy&quot;&gt;Dev Proxy&lt;/a&gt; to &lt;a href=&quot;https://learn.microsoft.com/microsoft-cloud/dev/dev-proxy/how-to/simulate-crud-api&quot;&gt;emulate CRUD APIs&lt;/a&gt; at the recent European Cloud Summit. While I was first suspecting an issue with Dev Proxy, it turned out to be a misconfiguration of the client that I was using to submit the request to the API.&lt;/p&gt;
  202. </description><pubDate>Mon, 16 Jun 2025 13:56:17 GMT</pubDate></item><item><title>Use Prompty with Foundry Local</title><link>https://blog.mastykarz.nl/prompty-foundry-local/</link><guid isPermaLink="true">https://blog.mastykarz.nl/prompty-foundry-local/</guid><description>&lt;p&gt;&lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2025/06/banner.png&quot; alt=&quot;Use Prompty with Foundry Local&quot; class=&quot;webfeedsFeaturedVisual&quot; /&gt;&lt;/p&gt;&lt;p&gt;&lt;a href=&quot;https://prompty.ai/&quot;&gt;Prompty&lt;/a&gt; is a powerful tool for managing prompts in AI applications. Not only does it allow you to easily test your prompts during development, but it also provides observability, understandability and portability. Here&apos;s how to use Prompty with Foundry Local to support your AI applications with on-device inference.&lt;/p&gt;
  203. &lt;h2&gt;Foundry Local&lt;/h2&gt;
  204. &lt;p&gt;At the Build &apos;25 conference, Microsoft announced &lt;a href=&quot;https://devblogs.microsoft.com/foundry/unlock-instant-on-device-ai-with-foundry-local/&quot;&gt;Foundry Local&lt;/a&gt;, a new tool that allows developers to run AI models locally on their devices. Foundry Local offers developers several benefits, including performance, privacy, and cost savings.&lt;/p&gt;
  205. &lt;h2&gt;Why Prompty?&lt;/h2&gt;
  206. &lt;p&gt;When you build AI applications with Foundry Local, but also other language model hosts, consider using Prompty to manage your prompts. With Prompty, you store your prompts in separate files, making it easy to test and adjust them without changing your code. Prompty also supports templating, allowing you to create dynamic prompts that adapt to different contexts or user inputs.&lt;/p&gt;
  207. &lt;h2&gt;Using Prompty with Foundry Local&lt;/h2&gt;
  208. &lt;p&gt;The most convenient way to use Prompty with Foundry Local is to create a new configuration for Foundry Local. Using a separate configuration allows you to seamlessly test your prompts without having to repeat the configuration for every prompt. It also allows you to easily switch between different configurations, such as Foundry Local and other language model hosts.&lt;/p&gt;
  209. &lt;h3&gt;Install Prompty and Foundry Local&lt;/h3&gt;
  210. &lt;p&gt;To get started, install the &lt;a href=&quot;https://marketplace.visualstudio.com/items?itemName=ms-toolsai.prompty&quot;&gt;Prompty Visual Studio Code extension&lt;/a&gt; and &lt;a href=&quot;https://learn.microsoft.com/en-us/azure/ai-foundry/foundry-local/get-started&quot;&gt;Foundry Local&lt;/a&gt;. Start Foundry Local from the command line by running &lt;code&gt;foundry service start&lt;/code&gt; and note the URL on which it listens for requests, such as &lt;code&gt;http://localhost:5272&lt;/code&gt; or &lt;code&gt;http://localhost:5273&lt;/code&gt;&lt;/p&gt;
  211. &lt;h3&gt;Create a new Prompty configuration for Foundry Local&lt;/h3&gt;
  212. &lt;p&gt;If you don&apos;t have a Prompty file yet, create one to easily access Prompty settings. In Visual Studio Code, open &lt;strong&gt;Explorer&lt;/strong&gt;, click right to open the context menu, and select &lt;strong&gt;New Prompty&lt;/strong&gt;. This creates a &lt;code&gt;basic.prompty&lt;/code&gt; file in your workspace.&lt;/p&gt;
  213. &lt;p&gt;&lt;picture&gt;
  214.    &lt;source srcset=&quot;https://blog.mastykarz.nl/assets/images/2025/06/prompty-file.webp&quot; type=&quot;image/webp&quot;&gt;
  215.    &lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2025/06/prompty-file.png&quot; alt=&quot;Visual Studio Code Explorer panel showing a newly created Prompty file named basic.prompty. The file is also opened in the editor.&quot;&gt;
  216. &lt;/picture&gt;&lt;/p&gt;
  217. &lt;h3&gt;Create the Foundry Local configuration&lt;/h3&gt;
  218. &lt;p&gt;From the status bar, select &lt;strong&gt;default&lt;/strong&gt; to open the Prompty configuration picker. When prompted to select the configuration, choose &lt;strong&gt;Add or Edit...&lt;/strong&gt;.&lt;/p&gt;
  219. &lt;p&gt;&lt;picture&gt;
  220.    &lt;source srcset=&quot;https://blog.mastykarz.nl/assets/images/2025/06/prompty-add-edit-configuration.webp&quot; type=&quot;image/webp&quot;&gt;
  221.    &lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2025/06/prompty-add-edit-configuration.png&quot; alt=&quot;Prompty configuration picker in the Visual Studio Code status bar, showing the default configurations and an option to add or edit configurations.&quot;&gt;
  222. &lt;/picture&gt;&lt;/p&gt;
  223. &lt;p&gt;In the settings pane, choose &lt;strong&gt;Edit in settings.json&lt;/strong&gt;.&lt;/p&gt;
  224. &lt;p&gt;&lt;picture&gt;
  225.    &lt;source srcset=&quot;https://blog.mastykarz.nl/assets/images/2025/06/prompty-edit-model-configurations-settings.webp&quot; type=&quot;image/webp&quot;&gt;
  226.    &lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2025/06/prompty-edit-model-configurations-settings.png&quot; alt=&quot;Visual Studio Code settings interface displaying the Prompty Model Configurations section. The interface shows options to configure model settings with an Edit in settings.json button prominently displayed.&quot;&gt;
  227. &lt;/picture&gt;&lt;/p&gt;
  228. &lt;p&gt;In the &lt;code&gt;settings.json&lt;/code&gt; file, to the &lt;strong&gt;prompty.modelConfigurations&lt;/strong&gt; collection, add a new configuration for Foundry Local, for example (ignore comments):&lt;/p&gt;
  229. &lt;pre&gt;&lt;code class=&quot;language-json&quot;&gt;{
  230.  // Foundry Local model ID that you want to use
  231.  &amp;quot;name&amp;quot;: &amp;quot;Phi-4-mini-instruct-generic-gpu&amp;quot;,
  232.  // API type; Foundry Local exposes OpenAI-compatible APIs
  233.  &amp;quot;type&amp;quot;: &amp;quot;openai&amp;quot;,
  234.  // API key required for the OpenAI SDK, but not used by Foundry Local
  235.  &amp;quot;api_key&amp;quot;: &amp;quot;local&amp;quot;,
  236.  // The URL where Foundry Local exposes its API
  237.  &amp;quot;base_url&amp;quot;: &amp;quot;http://localhost:5272/v1&amp;quot;
  238. }
  239. &lt;/code&gt;&lt;/pre&gt;
  240. &lt;blockquote&gt;
  241. &lt;p&gt;&lt;strong&gt;Important&lt;/strong&gt;: Be sure to check that you use the correct URL for Foundry Local. If you started Foundry Local with a different port, adjust the URL accordingly.&lt;/p&gt;
  242. &lt;/blockquote&gt;
  243. &lt;p&gt;Save your changes, and go back to the &lt;code&gt;.prompty&lt;/code&gt; file. Once again, select the &lt;strong&gt;default&lt;/strong&gt; configuration from the status bar, and from the list choose &lt;strong&gt;Phi-4-mini-instruct-generic-gpu&lt;/strong&gt;.&lt;/p&gt;
  244. &lt;p&gt;&lt;picture&gt;
  245.    &lt;source srcset=&quot;https://blog.mastykarz.nl/assets/images/2025/06/prompty-select-configuration.webp&quot; type=&quot;image/webp&quot;&gt;
  246.    &lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2025/06/prompty-select-configuration.png&quot; alt=&quot;Visual Studio Code dropdown menu displaying Prompty configuration options. The menu shows several configuration choices including default, Phi-4-mini-instruct-generic-gpu (which appears to be highlighted or selected), and other model configurations.&quot;&gt;
  247. &lt;/picture&gt;&lt;/p&gt;
  248. &lt;p&gt;Since the model and API are configured, you can remove them from the &lt;code&gt;.prompty&lt;/code&gt; file.&lt;/p&gt;
  249. &lt;p&gt;&lt;picture&gt;
  250.    &lt;source srcset=&quot;https://blog.mastykarz.nl/assets/images/2025/06/prompty-no-config.webp&quot; type=&quot;image/webp&quot;&gt;
  251.    &lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2025/06/prompty-no-config.png&quot; alt=&quot;Visual Studio Code editor displaying a basic.prompty file with the model and api sections removed from the configuration.&quot;&gt;
  252. &lt;/picture&gt;&lt;/p&gt;
  253. &lt;h3&gt;Test your prompts&lt;/h3&gt;
  254. &lt;p&gt;With the newly created Foundry Local configuration selected, in the &lt;code&gt;.prompty&lt;/code&gt; file, press &lt;kbd&gt;F5&lt;/kbd&gt; to test the prompt.&lt;/p&gt;
  255. &lt;p&gt;The first time you run the prompt, it may take a few seconds because Foundry Local needs to load the model.&lt;/p&gt;
  256. &lt;p&gt;&lt;picture&gt;
  257.    &lt;source srcset=&quot;https://blog.mastykarz.nl/assets/images/2025/06/prompty-running.webp&quot; type=&quot;image/webp&quot;&gt;
  258.    &lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2025/06/prompty-running.png&quot; alt=&quot;Visual Studio Code editor displaying notifcation message for the Prompty extension that it&apos;s using the Phi-4-mini-instruct-generic-gpu model configuration.&quot;&gt;
  259. &lt;/picture&gt;&lt;/p&gt;
  260. &lt;p&gt;Eventually, you should see the response from Foundry Local in the output pane.&lt;/p&gt;
  261. &lt;p&gt;&lt;picture&gt;
  262.    &lt;source srcset=&quot;https://blog.mastykarz.nl/assets/images/2025/06/prompty-foundry-local-output.webp&quot; type=&quot;image/webp&quot;&gt;
  263.    &lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2025/06/prompty-foundry-local-output.png&quot; alt=&quot;Visual Studio Code editor displaying the response from Foundry Local in the output panel.&quot;&gt;
  264. &lt;/picture&gt;&lt;/p&gt;
  265. &lt;h2&gt;Summary&lt;/h2&gt;
  266. &lt;p&gt;Using Prompty with Foundry Local allows you to easily manage and test your prompts while running AI models locally. By creating a dedicated Prompty configuration for Foundry Local, you can conveniently test your prompts with Foundry Local models and switch between different model hosts and models if needed.&lt;/p&gt;
  267. </description><pubDate>Fri, 13 Jun 2025 17:11:39 GMT</pubDate></item><item><title>Integrate Microsoft 365 Copilot declarative agents with Azure AI Search</title><link>https://blog.mastykarz.nl/integrate-microsoft-365-copilot-declarative-agents-azure-ai-search/</link><guid isPermaLink="true">https://blog.mastykarz.nl/integrate-microsoft-365-copilot-declarative-agents-azure-ai-search/</guid><description>&lt;p&gt;&lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2025/05/banner.png&quot; alt=&quot;Integrate Microsoft 365 Copilot declarative agents with Azure AI Search&quot; class=&quot;webfeedsFeaturedVisual&quot; /&gt;&lt;/p&gt;&lt;p&gt;Bringing knowledge to Microsoft 365 Copilot declarative agents using Azure AI Search is the one option that we don&apos;t talk about enough. And it&apos;s a shame, because it&apos;s a great way to strike the balance between having more control over indexing and relevance, without the complexity of standing up a custom engine agent with a language model deployment. Here&apos;s why.&lt;/p&gt;
  268. &lt;h2&gt;Know it all&lt;/h2&gt;
  269. &lt;p&gt;We all want our agents to know as much as possible. After all, the more they know, the more helpful they are, right? While the language models that back them keep improving, they&apos;re still far from including everything. A lot of the knowledge that we need agents to have is specific to our organization. Training a custom model on organizational knowledge, in most cases, just doesn&apos;t make sense. The information changes too often. It&apos;s also not business-critical enough to justify the cost of training a model for most. In many cases, knowledge items also need to be secured so that only those with the right permissions can see them. This is why we&apos;re resorting to RAG as the mechanism to bring additional knowledge to our agents.&lt;/p&gt;
  270. &lt;h2&gt;RAG on Microsoft 365 Copilot agents&lt;/h2&gt;
  271. &lt;p&gt;When building Microsoft 365 Copilot agents, we have several options for bringing additional knowledge to them.&lt;/p&gt;
  272. &lt;p&gt;&lt;picture&gt;
  273.  &lt;source srcset=&quot;https://blog.mastykarz.nl/assets/images/2025/05/m365-agent-rag.webp&quot; type=&quot;image/webp&quot;&gt;
  274.  &lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2025/05/m365-agent-rag.png&quot; alt=&quot;Different options for bringing additional knowledge to Microsoft 365 Copilot agents and their pros and cons&quot;&gt;
  275. &lt;/picture&gt;&lt;/p&gt;
  276. &lt;h3&gt;OneDrive/SharePoint&lt;/h3&gt;
  277. &lt;p&gt;The first, and the most obvious option, is to use OneDrive or SharePoint to store your knowledge items. It&apos;s a hands-off approach that just works. All you need to do is upload your files to a document library, and Microsoft 365 will take care of the rest. It&apos;ll automatically index the files and make them available to Microsoft 365 Copilot and its agents. It also automatically ensures that only those with the right permissions can see the files in agents&apos; responses. But this simplicity comes at a cost.&lt;/p&gt;
  278. &lt;p&gt;Microsoft 365 doesn&apos;t support indexing all file types. While it does support a lot of them, if your organization uses one that&apos;s not supported, Microsoft 365 won&apos;t index them, and there&apos;s nothing you can do about it.&lt;/p&gt;
  279. &lt;p&gt;You also don&apos;t get to choose how Microsoft 365 indexes your files. You simply upload files, and Microsoft 365 takes care of the rest. You have no control over how the files&apos; contents are parsed and how they&apos;re chunked.&lt;/p&gt;
  280. &lt;p&gt;Finally, you don&apos;t get to choose the relevance of the files. For most, that&apos;s a good thing, because let&apos;s face it: most of us aren&apos;t experts in relevance algorithms and search strategies. But if you understand your domain knowledge and have specific requirements for telling which knowledge items are more relevant than others, you might want to have more control over the relevance.&lt;/p&gt;
  281. &lt;h3&gt;Custom engine agents&lt;/h3&gt;
  282. &lt;p&gt;On the other side of the spectrum are custom engine agents. Building a custom engine agent means that you&apos;re bringing the full infrastructure: from the compute resource that hosts the agent to the language model that powers it. While it gives you full control over indexing additional data sources, tool usage, and orchestration, it also means additional complexity to ensure security and responsible use of AI. There&apos;s however a middle ground between the two extremes.&lt;/p&gt;
  283. &lt;h3&gt;Microsoft Graph connectors&lt;/h3&gt;
  284. &lt;p&gt;Another option at your disposal is to use Microsoft Graph connectors. Graph connectors ingest data from external systems into Microsoft 365. There are many connectors available in Microsoft 365 for you to use, and you can also build your own.&lt;/p&gt;
  285. &lt;p&gt;If your main challenge is having knowledge stored in unsupported data formats, then Graph connectors are a great option you should consider. Using Graph connectors, you basically create a push-based search index inside Microsoft 365. You define the schema for your data, specify which properties are searchable and filterable, and push the content, along with its access control list, to Microsoft 365.&lt;/p&gt;
  286. &lt;p&gt;Since you control pushing the content to Microsoft 365, you have some degree of control over how the content is indexed. It&apos;s your responsibility to extract the content from the source system and transform it into a structure compatible with Microsoft 365. This allows you to parse complex elements and include all the necessary information to convey their meaning. If your content items are large, you can also break them down into manageable chunks to help Microsoft 365 find the relevant information. For each chunk, you can also specify the URL to the original content, so that when users get a citation, they&apos;re pointed to the source system. There&apos;s, however, one thing that Graph connectors don&apos;t let you control.&lt;/p&gt;
  287. &lt;p&gt;When you ingest content into Microsoft 365 using Graph connectors, you don&apos;t get controls to define the relevance of the content. Microsoft 365 uses its predefined relevance algorithms to determine the relevance of the content. If you have specific requirements for relevance, the default settings might not give you the results you&apos;re looking for.&lt;/p&gt;
  288. &lt;h3&gt;Azure AI Search&lt;/h3&gt;
  289. &lt;p&gt;Often, we turn to building custom engine agents too soon, forgetting that we can use Azure AI Search to bring additional knowledge to Microsoft 365 Copilot agents. Azure AI Search is a managed search service that allows you to build search-based solutions. Similarly to Graph connectors, Azure AI Search allows you to have full control over indexing your content. But unlike other solutions, it also allows you to define custom ranking models. This means that you can define which information is the most relevant for your organization. As a bonus, you can decide whether you want to use lexical, semantic, vector, or hybrid search. This gives you the flexibility to choose the best search strategy for your organization and scenario.&lt;/p&gt;
  290. &lt;p&gt;Azure AI Search is a service on Microsoft Azure, and there is a cost involved with using it. Still, if your solution requires controlling context indexing and relevance, it&apos;s a great solution that you should consider before building a custom engine agent.&lt;/p&gt;
  291. &lt;h2&gt;Integrating declarative agents with Azure AI Search&lt;/h2&gt;
  292. &lt;p&gt;Integrating declarative agents with Azure AI Search is no different than &lt;a href=&quot;https://learn.microsoft.com/training/paths/copilot-microsoft-365-declarative-agents-api-plugins-visual-studio-code/&quot;&gt;integrating an API plugin&lt;/a&gt;. For the declarative agent, Azure AI Search is just another API. By default, Azure AI Search is secured with an API key. To integrate it with a declarative agent, register the API key in the Teams Developer Portal and include the registration ID in the agent&apos;s manifest. Then, you include the Azure AI Search&apos;s API spec in the project. Here&apos;s an example:&lt;/p&gt;
  293. &lt;pre&gt;&lt;code class=&quot;language-yaml&quot;&gt;openapi: 3.0.4
  294. info:
  295.  title: Knowledge base search
  296.  description: This API allows you to search for relevant knowledge in Azure AI Search indexes.
  297.  version: v1.0
  298. servers:
  299.  - url: https://${{AI_SEARCH_INSTANCE}}.search.windows.net
  300. security:
  301.  - apiKeyAuth: []
  302. paths:
  303.  /indexes/{index}/docs/search:
  304.    post:
  305.      x-openai-isConsequential: false
  306.      description: Search knowledge in Azure AI Search indexes.
  307.      operationId: searchKnowledge
  308.      security:
  309.        - apiKeyAuth: []
  310.      parameters:
  311.        - name: api-version
  312.          in: query
  313.          required: true
  314.          schema:
  315.            type: string
  316.            default: &amp;quot;2024-11-01-preview&amp;quot;
  317.        - name: index
  318.          in: path
  319.          required: true
  320.          schema:
  321.            type: string
  322.            default: &amp;quot;${{AI_SEARCH_INDEX}}&amp;quot;
  323.      requestBody:
  324.        content:
  325.          application/json:
  326.            schema:
  327.              type: object
  328.              properties:
  329.                search:
  330.                  type: string
  331.                count:
  332.                  type: boolean
  333.                  default: true
  334.                queryType:
  335.                  type: string
  336.                  default: semantic
  337.                semanticConfiguration:
  338.                  type: string
  339.                  default: knowledge
  340.                captions:
  341.                  type: string
  342.                  default: extractive
  343.                answers:
  344.                  type: string
  345.                  default: extractive|count-3
  346.                queryLanguage:
  347.                  type: string
  348.              required:
  349.                - search
  350.                - count
  351.                - queryType
  352.                - semanticConfiguration
  353.                - captions
  354.                - answers
  355.      responses:
  356.        &apos;200&apos;:
  357.          description: OK
  358.          content:
  359.            application/json:
  360.              schema:
  361.                type: string            
  362. components:
  363.  securitySchemes:
  364.    apiKeyAuth:
  365.      type: apiKey
  366.      in: header
  367.      name: api-key
  368.      description: API key for Azure AI Search
  369. &lt;/code&gt;&lt;/pre&gt;
  370. &lt;p&gt;Finally, describe the search API operation in an API plugin for the agent:&lt;/p&gt;
  371. &lt;pre&gt;&lt;code class=&quot;language-json&quot;&gt;{
  372.  &amp;quot;$schema&amp;quot;: &amp;quot;https://developer.microsoft.com/json-schemas/copilot/plugin/v2.2/schema.json&amp;quot;,
  373.  &amp;quot;schema_version&amp;quot;: &amp;quot;v2.2&amp;quot;,
  374.  &amp;quot;name_for_human&amp;quot;: &amp;quot;Knowledge base&amp;quot;,
  375.  &amp;quot;description_for_human&amp;quot;: &amp;quot;This plugin allows you to search knowledge in Azure AI Search indexes.&amp;quot;,
  376.  &amp;quot;description_for_model&amp;quot;: &amp;quot;This plugin allows you to search knowledge in Azure AI Search indexes.&amp;quot;,
  377.  &amp;quot;namespace&amp;quot;: &amp;quot;m365knowledge&amp;quot;,
  378.  &amp;quot;functions&amp;quot;: [
  379.    {
  380.      &amp;quot;name&amp;quot;: &amp;quot;searchKnowledge&amp;quot;,
  381.      &amp;quot;description&amp;quot;: &amp;quot;Search knowledge in Azure AI Search indexes.&amp;quot;,
  382.      &amp;quot;capabilities&amp;quot;: {
  383.        &amp;quot;response_semantics&amp;quot;: {
  384.          &amp;quot;data_path&amp;quot;: &amp;quot;$.value&amp;quot;,
  385.          &amp;quot;properties&amp;quot;: {
  386.            &amp;quot;title&amp;quot;: &amp;quot;title&amp;quot;,
  387.            &amp;quot;url&amp;quot;: &amp;quot;url&amp;quot;
  388.          },
  389.          &amp;quot;static_template&amp;quot;: {
  390.            &amp;quot;file&amp;quot;: &amp;quot;adaptiveCards/searchKnowledge.json&amp;quot;
  391.          }
  392.        }
  393.      }
  394.    }
  395.  ],
  396.  &amp;quot;runtimes&amp;quot;: [
  397.    {
  398.      &amp;quot;type&amp;quot;: &amp;quot;OpenApi&amp;quot;,
  399.      &amp;quot;auth&amp;quot;: {
  400.        &amp;quot;type&amp;quot;: &amp;quot;ApiKeyPluginVault&amp;quot;,
  401.        &amp;quot;reference_id&amp;quot;: &amp;quot;${{APIKEYAUTH_REGISTRATION_ID}}&amp;quot;
  402.      },
  403.      &amp;quot;spec&amp;quot;: {
  404.        &amp;quot;url&amp;quot;: &amp;quot;apiSpecificationFile/openapi.yaml&amp;quot;
  405.      },
  406.      &amp;quot;run_for_functions&amp;quot;: [
  407.        &amp;quot;searchKnowledge&amp;quot;
  408.      ]
  409.    }
  410.  ]
  411. }
  412. &lt;/code&gt;&lt;/pre&gt;
  413. &lt;h2&gt;Summary&lt;/h2&gt;
  414. &lt;p&gt;Bringing knowledge to Microsoft 365 Copilot declarative agents using Azure AI Search is a great way to strike the balance between having more control over indexing and relevance, without the complexity of standing up a custom engine agent with a language model deployment. It allows you to define custom ranking models and choose the best search strategy for your organization. If you&apos;re looking for a solution that gives you more control over indexing and relevance, consider using Azure AI Search before building a custom engine agent.&lt;/p&gt;
  415. </description><pubDate>Mon, 12 May 2025 14:21:08 GMT</pubDate></item><item><title>Calculate the number of language model tokens for a string</title><link>https://blog.mastykarz.nl/calculate-number-language-model-tokens-string/</link><guid isPermaLink="true">https://blog.mastykarz.nl/calculate-number-language-model-tokens-string/</guid><description>&lt;p&gt;&lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2025/01/banner.png&quot; alt=&quot;Calculate the number of language model tokens for a string&quot; class=&quot;webfeedsFeaturedVisual&quot; /&gt;&lt;/p&gt;&lt;p&gt;Here&apos;s an easy way to calculate the number of language model tokens for a string.&lt;/p&gt;
  416. &lt;p&gt;When working with language models, you might need to know how many tokens are in a string. Often, you use this information to estimate the cost for running your application. You might also need to know the number of tokens to tell if your text fits into the context window of the language model you&apos;re working with, or if you need to chunk it first.&lt;/p&gt;
  417. &lt;p&gt;You can roughly estimate the number of tokens by dividing the number of characters in your string by 4. This is however a &lt;em&gt;very rough estimate&lt;/em&gt;. In reality, the actual number of tokens strongly depends on the language model you&apos;re using. So for your calculations to be correct, you want to base them on the actual language model that you&apos;re using.&lt;/p&gt;
  418. &lt;p&gt;To help you calculate the number of tokens in a string, I put together a &lt;a href=&quot;https://github.com/waldekmastykarz/python-tokenize&quot;&gt;Jupyter Notebook&lt;/a&gt;. It allows you to calculate the number of tokens for a string, a file, or all files in a folder.&lt;/p&gt;
  419. &lt;p&gt;After you clone the repo, start by restoring all dependencies. You can do this using &lt;a href=&quot;https://docs.astral.sh/uv/&quot;&gt;uv&lt;/a&gt;, by running:&lt;/p&gt;
  420. &lt;pre&gt;&lt;code class=&quot;language-sh&quot;&gt;uv sync
  421. &lt;/code&gt;&lt;/pre&gt;
  422. &lt;p&gt;Next, open the notebook. Start with specifying the string/file/folder for which you want to calculate the number of tokens. Then, choose the language model you&apos;re using in your application. You can either use a Hugging Face or OpenAI model. For Hugging Face models, specify the name in the &lt;code&gt;user/model_name&lt;/code&gt; format. For OpenAI models, simply pass the name of the model. Here&apos;s the &lt;a href=&quot;https://github.com/openai/tiktoken/blob/63527649963def8c759b0f91f2eb69a40934e468/tiktoken/model.py#L22-L72&quot;&gt;list of supported models&lt;/a&gt;.&lt;/p&gt;
  423. &lt;p&gt;&lt;picture&gt;
  424.  &lt;source srcset=&quot;https://blog.mastykarz.nl/assets/images/2025/01/notebook-configuration.webp&quot; type=&quot;image/webp&quot;&gt;
  425.  &lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2025/01/notebook-configuration.png&quot; alt=&quot;Screenshot of a cell in a Jupyter notebook showing the configuration options&quot;&gt;
  426. &lt;/picture&gt;&lt;/p&gt;
  427. &lt;p&gt;Finally, run all cells to see the number of tokens for your string/file/folder. Scroll down to the respective cell to see the output.&lt;/p&gt;
  428. &lt;p&gt;&lt;picture&gt;
  429.  &lt;source srcset=&quot;https://blog.mastykarz.nl/assets/images/2025/01/number-tokens-text.webp&quot; type=&quot;image/webp&quot;&gt;
  430.  &lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2025/01/number-tokens-text.png&quot; alt=&quot;Output of a cell in a Jupyter notebook showing the number of tokens for a string&quot;&gt;
  431. &lt;/picture&gt;&lt;/p&gt;
  432. &lt;p&gt;What&apos;s great about calculating the number of tokens used in a string using a Jupyter Notebook is that it&apos;s running on your machine. It doesn&apos;t depend on an external service, it doesn&apos;t incur any costs and allows you to calculate the number of tokens for your strings in a secure way for virtually any language model.&lt;/p&gt;
  433. &lt;p&gt;Try it and let me know what you think!&lt;/p&gt;
  434. </description><pubDate>Mon, 20 Jan 2025 12:07:40 GMT</pubDate></item><item><title>Create npm package with CommonJS and ESM support in TypeScript</title><link>https://blog.mastykarz.nl/create-npm-package-commonjs-esm-typescript/</link><guid isPermaLink="true">https://blog.mastykarz.nl/create-npm-package-commonjs-esm-typescript/</guid><description>&lt;p&gt;&lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2024/06/npm-cjs-esm.png&quot; alt=&quot;Create npm package with CommonJS and ESM support in TypeScript&quot; class=&quot;webfeedsFeaturedVisual&quot; /&gt;&lt;/p&gt;&lt;p&gt;If you want to create an npm package and ensure it can be used by everyone, you&apos;ll want it to support CommonJS (CJS) and ECMAScript Modules (ESM). Here&apos;s how to build such a package using TypeScript.&lt;/p&gt;
  435. &lt;h2&gt;CommonJS and ESM&lt;/h2&gt;
  436. &lt;p&gt;When building JavaScript apps, you have two module systems to choose from: CommonJS and ECMAScript Modules. Despite the recent rise of ESM, CommonJS is still widely used, and not to mention default in Node.js. To make sure your npm package can be used by everyone, you&apos;ll want to support both module systems.&lt;/p&gt;
  437. &lt;h2&gt;Project setup&lt;/h2&gt;
  438. &lt;p&gt;CommonJS and ESM are not compatible with each other. To support both, your package needs to contain two versions of the code, one for CommonJS and one for ESM. To support usage in TypeScript, you&apos;ll also need to include type definitions. Since the shape of the exported API is the same, you can use the same TypeScript code for both CommonJS and ESM.&lt;/p&gt;
  439. &lt;p&gt;Here&apos;s the global structure of the code that your package should produce that you&apos;re after:&lt;/p&gt;
  440. &lt;pre&gt;&lt;code class=&quot;language-plaintext&quot;&gt;project
  441. ├── dist // package output
  442. │   ├── cjs
  443. │   │   └── CJS code
  444. │   ├── esm
  445. │   │   └── ESM code
  446. │   └── types
  447. │       └── TypeScript type definitions
  448. └── src // source code
  449.    └── TypeScript source
  450. &lt;/code&gt;&lt;/pre&gt;
  451. &lt;p&gt;The following sections describe how to set up the project to produce such output.&lt;/p&gt;
  452. &lt;blockquote&gt;
  453. &lt;p&gt;Note: Check out the full source code of the &lt;a href=&quot;https://github.com/waldekmastykarz/node-ts-cjs-esm&quot;&gt;sample project&lt;/a&gt; on GitHub.&lt;/p&gt;
  454. &lt;/blockquote&gt;
  455. &lt;h3&gt;TypeScript configuration&lt;/h3&gt;
  456. &lt;p&gt;TypeScript can produce only one output at a time. To produce both CommonJS and ESM code, you need two different TypeScript configuration files (&lt;code&gt;tsconfig.json&lt;/code&gt;). The good news is, that you can reuse common settings and only specify the distinct parts for CJS and ESM.&lt;/p&gt;
  457. &lt;p&gt;Start, by creating a base TypeScript configuration file with shared settings (&lt;code&gt;tsconfig.base.json&lt;/code&gt;):&lt;/p&gt;
  458. &lt;pre&gt;&lt;code class=&quot;language-json&quot;&gt;{
  459.  &amp;quot;compilerOptions&amp;quot;: {
  460.    &amp;quot;lib&amp;quot;: [
  461.      &amp;quot;esnext&amp;quot;
  462.    ],
  463.    &amp;quot;declaration&amp;quot;: true,
  464.    &amp;quot;declarationDir&amp;quot;: &amp;quot;./dist/types&amp;quot;,
  465.    &amp;quot;strict&amp;quot;: true,
  466.    &amp;quot;esModuleInterop&amp;quot;: true,
  467.    &amp;quot;skipLibCheck&amp;quot;: true,
  468.    &amp;quot;forceConsistentCasingInFileNames&amp;quot;: true,
  469.    &amp;quot;moduleResolution&amp;quot;: &amp;quot;node&amp;quot;,
  470.    &amp;quot;baseUrl&amp;quot;: &amp;quot;.&amp;quot;,
  471.    &amp;quot;rootDir&amp;quot;: &amp;quot;./src&amp;quot;
  472.  },
  473.  &amp;quot;include&amp;quot;: [
  474.    &amp;quot;src&amp;quot;
  475.  ],
  476.  &amp;quot;exclude&amp;quot;: [
  477.    &amp;quot;dist&amp;quot;,
  478.    &amp;quot;node_modules&amp;quot;
  479.  ]
  480. }
  481. &lt;/code&gt;&lt;/pre&gt;
  482. &lt;p&gt;The file specifies, among other things, the location of the source code, folders to exclude, and the output directory for type definitions.&lt;/p&gt;
  483. &lt;p&gt;Next, create two separate configuration files for CommonJS and ESM.&lt;/p&gt;
  484. &lt;p&gt;&lt;code&gt;tsconfig.cjs.json&lt;/code&gt;:&lt;/p&gt;
  485. &lt;pre&gt;&lt;code class=&quot;language-json&quot;&gt;{
  486.  &amp;quot;extends&amp;quot;: &amp;quot;./tsconfig.base.json&amp;quot;,
  487.  &amp;quot;compilerOptions&amp;quot;: {
  488.    &amp;quot;module&amp;quot;: &amp;quot;commonjs&amp;quot;,
  489.    &amp;quot;outDir&amp;quot;: &amp;quot;./dist/cjs&amp;quot;,
  490.    &amp;quot;target&amp;quot;: &amp;quot;ES2015&amp;quot;
  491.  }
  492. }
  493. &lt;/code&gt;&lt;/pre&gt;
  494. &lt;p&gt;&lt;code&gt;tsconfig.esm.json&lt;/code&gt;:&lt;/p&gt;
  495. &lt;pre&gt;&lt;code class=&quot;language-json&quot;&gt;{
  496.  &amp;quot;extends&amp;quot;: &amp;quot;./tsconfig.base.json&amp;quot;,
  497.  &amp;quot;compilerOptions&amp;quot;: {
  498.    &amp;quot;module&amp;quot;: &amp;quot;esnext&amp;quot;,
  499.    &amp;quot;outDir&amp;quot;: &amp;quot;./dist/esm&amp;quot;,
  500.    &amp;quot;target&amp;quot;: &amp;quot;esnext&amp;quot;
  501.  }
  502. }
  503. &lt;/code&gt;&lt;/pre&gt;
  504. &lt;p&gt;Notice, how both files extend the base configuration and only specify the module system and the distinct output directory.&lt;/p&gt;
  505. &lt;h3&gt;package.json configuration&lt;/h3&gt;
  506. &lt;p&gt;After setting up TypeScript, continue with the &lt;code&gt;package.json&lt;/code&gt; configuration. Here, you need to do a few things. You need to specify entry points for CJS and ESM consumers and define scripts to build the package in both formats.&lt;/p&gt;
  507. &lt;h4&gt;Define entry points&lt;/h4&gt;
  508. &lt;p&gt;Start, by defining entry points for CJS and ESM consumers.&lt;/p&gt;
  509. &lt;pre&gt;&lt;code class=&quot;language-json&quot;&gt;{
  510.  &amp;quot;name&amp;quot;: &amp;quot;ts-cjs-esm&amp;quot;,
  511.  &amp;quot;version&amp;quot;: &amp;quot;1.0.0&amp;quot;,
  512.  &amp;quot;main&amp;quot;: &amp;quot;./dist/cjs/index.js&amp;quot;,
  513.  &amp;quot;module&amp;quot;: &amp;quot;./dist/esm/index.mjs&amp;quot;,
  514.  &amp;quot;types&amp;quot;: &amp;quot;./dist/types/index.d.ts&amp;quot;,
  515.  &amp;quot;files&amp;quot;: [
  516.    &amp;quot;dist&amp;quot;
  517.  ]
  518.  // ...
  519. }
  520. &lt;/code&gt;&lt;/pre&gt;
  521. &lt;p&gt;You&apos;ve got the main entry point to refer to the CJS code, and the module entry point to ESM code. Both formats reuse the same TypeScript type definitions. Unfortunately, the &lt;code&gt;main&lt;/code&gt; property overrules the &lt;code&gt;module&lt;/code&gt; property in Node.js, so you need an extra way to specify how different consumers should load the library. You can do this using the &lt;code&gt;exports&lt;/code&gt; property.&lt;/p&gt;
  522. &lt;pre&gt;&lt;code class=&quot;language-json&quot;&gt;{
  523.  &amp;quot;name&amp;quot;: &amp;quot;ts-cjs-esm&amp;quot;,
  524.  &amp;quot;version&amp;quot;: &amp;quot;1.0.0&amp;quot;,
  525.  &amp;quot;main&amp;quot;: &amp;quot;./dist/cjs/index.js&amp;quot;,
  526.  &amp;quot;module&amp;quot;: &amp;quot;./dist/esm/index.mjs&amp;quot;,
  527.  &amp;quot;types&amp;quot;: &amp;quot;./dist/types/index.d.ts&amp;quot;,
  528.  &amp;quot;exports&amp;quot;: {
  529.    &amp;quot;.&amp;quot;: {
  530.      &amp;quot;require&amp;quot;: &amp;quot;./dist/cjs/index.js&amp;quot;,
  531.      &amp;quot;import&amp;quot;: &amp;quot;./dist/esm/index.mjs&amp;quot;
  532.    }
  533.  },
  534.  &amp;quot;files&amp;quot;: [
  535.    &amp;quot;dist&amp;quot;
  536.  ]
  537.  // ...
  538. }
  539. &lt;/code&gt;&lt;/pre&gt;
  540. &lt;p&gt;Using the &lt;code&gt;exports&lt;/code&gt; property, you can specify how different consumers should load the library. The &lt;code&gt;require&lt;/code&gt; property points to the CJS code and the &lt;code&gt;import&lt;/code&gt; property to the ESM code.&lt;/p&gt;
  541. &lt;h4&gt;.mjs vs .js file extension&lt;/h4&gt;
  542. &lt;p&gt;You might&apos;ve noticed, that the ESM code has the &lt;code&gt;.mjs&lt;/code&gt; extension while CJS uses &lt;code&gt;.js&lt;/code&gt;. This is necessary to distinguish ESM from CJS code. If your package contains only ESM code, in the &lt;code&gt;package.json&lt;/code&gt; file you can configure the &lt;code&gt;type&lt;/code&gt; property to &lt;code&gt;module&lt;/code&gt; to indicate that the package contains ESM code. This instructs Node.js to treat all files as ES modules. Unfortunately, in this case, that&apos;s not possible, because the package contains both CJS and ESM code. This means, that you need to use the &lt;code&gt;.mjs&lt;/code&gt; extension to designate ESM files. One more thing that complicates matters some more is, that TypeScript doesn&apos;t allow you to specify the output file extension and always produces &lt;code&gt;.js&lt;/code&gt; files. To work around this, you need to rename the output files, and imports after the build.&lt;/p&gt;
  543. &lt;h4&gt;Define npm scripts&lt;/h4&gt;
  544. &lt;p&gt;With the basic package setup in place, let&apos;s continue with defining a few scripts to build the package in both formats.&lt;/p&gt;
  545. &lt;pre&gt;&lt;code class=&quot;language-json&quot;&gt;{
  546.  &amp;quot;name&amp;quot;: &amp;quot;ts-cjs-esm&amp;quot;,
  547.  // ...
  548.  &amp;quot;scripts&amp;quot;: {
  549.    &amp;quot;build:cjs&amp;quot;: &amp;quot;tsc -p tsconfig.cjs.json&amp;quot;,
  550.    &amp;quot;build:esm&amp;quot;: &amp;quot;tsc -p tsconfig.esm.json &amp;amp;&amp;amp; npm run rename:esm&amp;quot;,
  551.    &amp;quot;build&amp;quot;: &amp;quot;npm run build:cjs &amp;amp;&amp;amp; npm run build:esm&amp;quot;,
  552.    &amp;quot;clean&amp;quot;: &amp;quot;rimraf dist&amp;quot;,
  553.    &amp;quot;rename:esm&amp;quot;: &amp;quot;/bin/zsh ./scripts/fix-mjs.sh&amp;quot;,
  554.    &amp;quot;prepack&amp;quot;: &amp;quot;npm run clean &amp;amp;&amp;amp; npm run build&amp;quot;
  555.  }
  556.  // ...
  557. }
  558. &lt;/code&gt;&lt;/pre&gt;
  559. &lt;p&gt;We start with two build scripts, one for CJS and one for ESM, each pointing to the respective TypeScript config file. As mentioned previously, TypeScript doesn&apos;t allow you to specify the output file extension, so you need to rename the output files after the build. The &lt;code&gt;rename:esm&lt;/code&gt; script runs a shell script that renames the &lt;code&gt;.js&lt;/code&gt; files to &lt;code&gt;.mjs&lt;/code&gt; and updates import references. For convenience, we also include a script to clean the output directory and build the package before publishing.&lt;/p&gt;
  560. &lt;h4&gt;Install dependencies&lt;/h4&gt;
  561. &lt;p&gt;In npm scripts, we&apos;re using TypeScript and rimraf. Make sure to install them as dev dependencies:&lt;/p&gt;
  562. &lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;npm install --save-dev typescript rimraf
  563. &lt;/code&gt;&lt;/pre&gt;
  564. &lt;h3&gt;Helper scripts&lt;/h3&gt;
  565. &lt;p&gt;To support building ESM code, you need a helper script that renames the output files and updates import references. Create a shell script &lt;code&gt;fix-mjs.sh&lt;/code&gt; in the &lt;code&gt;scripts&lt;/code&gt; folder:&lt;/p&gt;
  566. &lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;for file in ./dist/esm/*.js; do
  567.  echo &amp;quot;Updating $file contents...&amp;quot;
  568.  sed -i &apos;&apos; &amp;quot;s/\.js&apos;/\.mjs&apos;/g&amp;quot; &amp;quot;$file&amp;quot;
  569.  echo &amp;quot;Renaming $file to ${file%.js}.mjs...&amp;quot;
  570.  mv &amp;quot;$file&amp;quot; &amp;quot;${file%.js}.mjs&amp;quot;
  571. done
  572. &lt;/code&gt;&lt;/pre&gt;
  573. &lt;p&gt;This script iterates over all &lt;code&gt;.js&lt;/code&gt; files in the &lt;code&gt;dist/esm&lt;/code&gt; folder. For each file, it replaces &lt;code&gt;.js&apos;&lt;/code&gt; with &lt;code&gt;.mjs&apos;&lt;/code&gt; in the file contents, and then renames the file to have the &lt;code&gt;.mjs&lt;/code&gt; extension.&lt;/p&gt;
  574. &lt;h3&gt;Git- and npmignore&lt;/h3&gt;
  575. &lt;p&gt;If you intend to store your source in a git repo, add a &lt;code&gt;.gitignore&lt;/code&gt; file to your project, to avoid including unnecessary files in the repository:&lt;/p&gt;
  576. &lt;pre&gt;&lt;code class=&quot;language-plaintext&quot;&gt;dist
  577. node_modules
  578. &lt;/code&gt;&lt;/pre&gt;
  579. &lt;p&gt;Since you&apos;re building a library, which you&apos;ll likely distribute, you should also add a &lt;code&gt;.npmignore&lt;/code&gt; file to exclude unnecessary files from the npm package:&lt;/p&gt;
  580. &lt;pre&gt;&lt;code class=&quot;language-plaintext&quot;&gt;scripts
  581. src
  582. &lt;/code&gt;&lt;/pre&gt;
  583. &lt;h3&gt;Sample source&lt;/h3&gt;
  584. &lt;p&gt;To test the setup, create simple TypeScript source files in the &lt;code&gt;src&lt;/code&gt; folder.&lt;/p&gt;
  585. &lt;p&gt;&lt;code&gt;myModule.ts&lt;/code&gt;:&lt;/p&gt;
  586. &lt;pre&gt;&lt;code class=&quot;language-typescript&quot;&gt;export function myFunction() {
  587.  return &apos;Hello World!&apos;;
  588. }
  589. &lt;/code&gt;&lt;/pre&gt;
  590. &lt;p&gt;&lt;code&gt;index.ts&lt;/code&gt;:&lt;/p&gt;
  591. &lt;pre&gt;&lt;code class=&quot;language-typescript&quot;&gt;export * from &apos;./myModule.js&apos;;
  592. &lt;/code&gt;&lt;/pre&gt;
  593. &lt;p&gt;Notice, that when referring to the &lt;code&gt;myModule&lt;/code&gt; file, you use the &lt;code&gt;.js&lt;/code&gt; extension. This is required for the ESM build to work correctly. When building the ESM package, the helper script updates the extension to &lt;code&gt;.mjs&lt;/code&gt;.&lt;/p&gt;
  594. &lt;h2&gt;Build the package&lt;/h2&gt;
  595. &lt;p&gt;To verify that the setup works, run the build script:&lt;/p&gt;
  596. &lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;npm run build
  597. &lt;/code&gt;&lt;/pre&gt;
  598. &lt;p&gt;If all is well, you should see the output in the &lt;code&gt;dist&lt;/code&gt; folder:&lt;/p&gt;
  599. &lt;pre&gt;&lt;code class=&quot;language-plaintext&quot;&gt;dist
  600. ├── cjs
  601. │   ├── index.js
  602. │   └── myModule.js
  603. ├── esm
  604. │   ├── index.mjs
  605. │   └── myModule.mjs
  606. └── types
  607.    ├── index.d.ts
  608.    └── myModule.d.ts
  609. &lt;/code&gt;&lt;/pre&gt;
  610. &lt;h2&gt;Summary&lt;/h2&gt;
  611. &lt;p&gt;When building an npm package, consider supporting both CommonJS and ECMAScript Modules so that your package can be used by everyone. To support CJS and ESM, you need to produce two versions of the code, one for each module system. By configuring your project in a specific way, you can produce a package that supports both module systems and includes TypeScript type definitions. This way, you can provide a seamless experience for everyone using your package, regardless of the module system they use.&lt;/p&gt;
  612. </description><pubDate>Sat, 29 Jun 2024 14:44:13 GMT</pubDate></item><item><title>Trace the location of API requests</title><link>https://blog.mastykarz.nl/trace-location-api-requests/</link><guid isPermaLink="true">https://blog.mastykarz.nl/trace-location-api-requests/</guid><description>&lt;p&gt;&lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2024/06/request-headers-src.png&quot; alt=&quot;Trace the location of API requests&quot; class=&quot;webfeedsFeaturedVisual&quot; /&gt;&lt;/p&gt;&lt;p&gt;These days, it&apos;s hard to imagine an app that&apos;s not using APIs. APIs give us access to data and insights and allow us to integrate with cloud services easily. But as we use more and more APIs, we struggle with debugging our app. What if a request fails? What if we need to change something about a particular request? How do we know where in our app the request is coming from?&lt;/p&gt;
  613. &lt;h2&gt;Tracing API requests&lt;/h2&gt;
  614. &lt;p&gt;If you call just a handful of API endpoints, and they&apos;re distinct to the specific portions of your app, it&apos;s easy to find where the request is coming from. But as your app grows, and you have more and more API requests, it becomes harder to trace the location of each request. It becomes even harder when you use SDKs or libraries and can&apos;t search for a specific API URL in your code. What if you could automatically trace the location of each API request in your app? What if each request contained information about where in your code it was called from?&lt;/p&gt;
  615. &lt;h2&gt;Tracing API requests in .NET&lt;/h2&gt;
  616. &lt;p&gt;To trace the location of API requests in .NET, you need a custom delegating handler. This handler automatically adds information about the location in your code where each API request has been called.&lt;/p&gt;
  617. &lt;pre&gt;&lt;code class=&quot;language-csharp&quot;&gt;using System.Diagnostics;
  618.  
  619. public class CodeLocationDelegatingHandler(HttpMessageHandler innerHandler) : DelegatingHandler(innerHandler)
  620. {
  621.    protected override Task&amp;lt;HttpResponseMessage&amp;gt; SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
  622.    {
  623.        // using Ben.Demystifier to get clearer information
  624.        var enhTrace = EnhancedStackTrace.Current();
  625.        // skip 3 frames which include this delegating handler and preparing
  626.        // the request
  627.        var frame = (EnhancedStackFrame)enhTrace.GetFrame(3);
  628.        var method = frame.MethodInfo.Name;
  629.        // or using System.Diagnostics.StackTrace, but it&apos;s less clear
  630.        // var frame = new StackFrame(0, true);
  631.        // var method = frame.GetMethod()?.Name ?? string.Empty;
  632.        var fileName = frame.GetFileName();
  633.        var lineNumber = frame.GetFileLineNumber();
  634.  
  635.        request.Headers.Add(&amp;quot;x-src-method&amp;quot;, method);
  636.        request.Headers.Add(&amp;quot;x-src&amp;quot;, $&amp;quot;{fileName}:{lineNumber}&amp;quot;);
  637.  
  638.        return base.SendAsync(request, cancellationToken);
  639.    }
  640. }
  641. &lt;/code&gt;&lt;/pre&gt;
  642. &lt;p&gt;Then, you can use this handler in your HttpClient:&lt;/p&gt;
  643. &lt;pre&gt;&lt;code class=&quot;language-csharp&quot;&gt;var handler = new CodeLocationDelegatingHandler(new HttpClientHandler());
  644. using var httpClient = new HttpClient(handler);
  645.  
  646. async Task Method1()
  647. {
  648.    var response = await httpClient.GetStringAsync(&amp;quot;https://jsonplaceholder.typicode.com/posts&amp;quot;);
  649.    Console.WriteLine(response);
  650. }
  651.  
  652. await Method1();
  653.  
  654. // The API request contains the following headers:
  655. // x-src-method: &amp;lt;&amp;lt;Main&amp;gt;$&amp;gt;g__Method1|0
  656. // x-src: C:\MyApp\Program.cs:6
  657. &lt;/code&gt;&lt;/pre&gt;
  658. &lt;p&gt;If you looked closely at the code, you might have noticed that we&apos;re using the &lt;a href=&quot;https://github.com/benaadams/Ben.Demystifier&quot;&gt;Ben.Demystifier&lt;/a&gt; package. This package helps us get clearer information about the stack trace. We need it because when you use async methods, .NET generates a lot of compiler-generated code, which makes it hard to understand where the request was called from. Also, keep in mind, that the ability to map the location of the request to the specific line in your code depends on the debug symbols (PDB) being available on runtime.&lt;/p&gt;
  659. &lt;h2&gt;Tracing API requests in JavaScript&lt;/h2&gt;
  660. &lt;p&gt;In JavaScript, to get the information about the location of the API request, you need to use the stack trace. You can get the stack trace by creating an Error object and reading its stack property. Once you have the location in your code, add it to the request headers. How you do it depends on the library you use to make API requests. For example, if you&apos;re using fetch, you can add the location information to the request headers:&lt;/p&gt;
  661. &lt;pre&gt;&lt;code class=&quot;language-javascript&quot;&gt;function getCallerLocation(error) {
  662.  const stack = error.stack.split(&apos;\n&apos;);
  663.  // skip the error message and the current function
  664.  const caller = stack[2].trim();
  665.  const regex = /at (\w+) \(([^)]+)\)/;
  666.  const match = regex.exec(caller);
  667.  const functionName = match[1];
  668.  const filePathLocation = match[2];
  669.  return { functionName, filePathLocation };
  670. }
  671.  
  672. async function codeLocationFetch(url, options = {}) {
  673.  const { functionName, filePathLocation } = getCallerLocation(new Error());
  674.  
  675.  const defaultHeaders = {
  676.    &apos;x-src-method&apos;: functionName,
  677.    &apos;x-src&apos;: filePathLocation
  678.  };
  679.  
  680.  const mergedOptions = {
  681.    ...options,
  682.    headers: {
  683.      ...defaultHeaders,
  684.      ...options.headers,
  685.    },
  686.  };
  687.  
  688.  return fetch(url, mergedOptions);
  689. }
  690.  
  691. async function func1() {
  692.  const res = await codeLocationFetch(&apos;https://jsonplaceholder.typicode.com/posts&apos;);
  693.  const json = await res.json();
  694.  console.log(json);
  695. }
  696.  
  697. await func1();
  698.  
  699. // The API request contains the following headers:
  700. // x-src-method: func1
  701. // x-src: file:///path/to/myapp/index.js:38:15
  702. &lt;/code&gt;&lt;/pre&gt;
  703. &lt;h2&gt;Privacy&lt;/h2&gt;
  704. &lt;p&gt;If you don&apos;t want to send the information about your code base to third parties, you could instrument your logging infrastructure to log the information about the source of the request, but then remove it from the request before sending it to the API. This way, you can still trace the location of the request in your logs, but you don&apos;t have to send the information to the API.&lt;/p&gt;
  705. &lt;h2&gt;Summary&lt;/h2&gt;
  706. &lt;p&gt;Including the information about where in your code each API request is called helps you trace the location of the request. This is invaluable when you&apos;re debugging your app and need to locate the specific API request in your codebase. By automatically adding the information about the location of the request to requests, you minimize the impact on your development process and ensure that the information is consistently applied across the whole app.&lt;/p&gt;
  707. </description><pubDate>Mon, 24 Jun 2024 11:02:34 GMT</pubDate></item><item><title>Add background color to menu bar icon in macOS</title><link>https://blog.mastykarz.nl/add-background-color-menu-bar-icon-macos/</link><guid isPermaLink="true">https://blog.mastykarz.nl/add-background-color-menu-bar-icon-macos/</guid><description>&lt;p&gt;&lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2024/02/proxystat-proxy-on.png&quot; alt=&quot;Add background color to menu bar icon in macOS&quot; class=&quot;webfeedsFeaturedVisual&quot; /&gt;&lt;/p&gt;&lt;p&gt;When building apps for macOS, you might want to add a background color to your icon in the menu bar. This is a great way to make your app stand out and make it easier for users to find your app in the menu bar.&lt;/p&gt;
  708. &lt;p&gt;To add a background color to your icon in the menu bar, the trick is to find the right view to apply the background color to. Here&apos;s how you can do it in Swift.&lt;/p&gt;
  709. &lt;p&gt;Start, by getting a reference to the status item. Then, get the status item&apos;s button. Configure the image to fit the button.&lt;/p&gt;
  710. &lt;pre&gt;&lt;code class=&quot;language-swift&quot;&gt;// get a reference to the status item
  711. let statusItem = NSStatusBar.system.statusItem(withLength: NSStatusItem.squareLength)
  712. // get the status item&apos;s button
  713. let button = statusItem.button
  714. // configure the image to fit the button
  715. button?.imageScaling = NSImageScaling.scaleProportionallyUpOrDown
  716. &lt;/code&gt;&lt;/pre&gt;
  717. &lt;p&gt;Next, get the view behind the button and apply the background color to it. Set the image on the button. To align the button with standard macOS buttons such as microphone, camera or screen recording, configure the corners to be rounded.&lt;/p&gt;
  718. &lt;pre&gt;&lt;code class=&quot;language-swift&quot;&gt;// this is the view you need to get
  719. let buttonView = button?.superview?.window?.contentView
  720. // set the image on the button
  721. button.image = NSImage(named:NSImage.Name(&amp;quot;ProxyEnabledWhite&amp;quot;))
  722. // set the background color
  723. buttonView.layer?.backgroundColor = NSColor.systemOrange.cgColor
  724. // round the corners
  725. buttonView.layer?.cornerRadius = 4
  726. &lt;/code&gt;&lt;/pre&gt;
  727. &lt;p&gt;If you get the wrong view, the background color will overlay the image. Also, configure the &lt;code&gt;template-rendering-intent&lt;/code&gt; of the image to show on the button as &lt;code&gt;original&lt;/code&gt;. This will ensure that the image is not affected by the background color and has enough contrast.&lt;/p&gt;
  728. &lt;p&gt;Here&apos;s the result:&lt;/p&gt;
  729. &lt;p&gt;&lt;picture&gt;
  730.  &lt;source srcset=&quot;https://blog.mastykarz.nl/assets/images/2024/02/proxystat-proxy-on.webp&quot; type=&quot;image/webp&quot;&gt;
  731.  &lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2024/02/proxystat-proxy-on.png&quot; alt=&quot;ProxyStat showing an orange icon when system proxy is configured&quot;&gt;
  732. &lt;/picture&gt;&lt;/p&gt;
  733. &lt;p&gt;To clear the background color, change the &lt;code&gt;backgroundColor&lt;/code&gt; to &lt;code&gt;NSColor.clear&lt;/code&gt;. Change the image configure as a template image so that it automatically adjusts to the system appearance. If you need, you can also choose to display the image as disabled.&lt;/p&gt;
  734. &lt;pre&gt;&lt;code class=&quot;language-swift&quot;&gt;buttonView.layer?.backgroundColor = NSColor.clear.cgColor
  735. button.image = NSImage(named:NSImage.Name(&amp;quot;ProxyEnabledBlack&amp;quot;))
  736. button.appearsDisabled = true
  737. &lt;/code&gt;&lt;/pre&gt;
  738. &lt;p&gt;Here&apos;s the result:&lt;/p&gt;
  739. &lt;p&gt;&lt;picture&gt;
  740.  &lt;source srcset=&quot;https://blog.mastykarz.nl/assets/images/2024/02/proxystat-proxy-off.webp&quot; type=&quot;image/webp&quot;&gt;
  741.  &lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2024/02/proxystat-proxy-off.png&quot; alt=&quot;ProxyStat showing a greyed out icon when system proxy is not configured&quot;&gt;
  742. &lt;/picture&gt;&lt;/p&gt;
  743. &lt;h2&gt;Summary&lt;/h2&gt;
  744. &lt;p&gt;Adding a background color to your icon in the menu bar on macOS is a great way to make your app stand out and make it easier for users to find your app in the menu bar. By applying the background color to the right view, you can combine it with your icon and align it with standard macOS buttons such as microphone, camera or screen recording.&lt;/p&gt;
  745. </description><pubDate>Sun, 04 Feb 2024 11:33:54 GMT</pubDate></item><item><title>Easily see if you have system proxy configured on macOS</title><link>https://blog.mastykarz.nl/easily-see-system-proxy-configured-macos/</link><guid isPermaLink="true">https://blog.mastykarz.nl/easily-see-system-proxy-configured-macos/</guid><description>&lt;p&gt;&lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2024/02/proxystat-proxy-on.png&quot; alt=&quot;Easily see if you have system proxy configured on macOS&quot; class=&quot;webfeedsFeaturedVisual&quot; /&gt;&lt;/p&gt;&lt;p&gt;ProxyStat is a macOS utility that shows when you have a system proxy configured. When you enable a system proxy, the ProxyStat icon the menu bar turns orange.&lt;/p&gt;
  746. &lt;p&gt;&lt;picture&gt;
  747.  &lt;source srcset=&quot;https://blog.mastykarz.nl/assets/images/2024/02/proxystat-proxy-on.webp&quot; type=&quot;image/webp&quot;&gt;
  748.  &lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2024/02/proxystat-proxy-on.png&quot; alt=&quot;ProxyStat showing an orange icon when system proxy is configured&quot;&gt;
  749. &lt;/picture&gt;&lt;/p&gt;
  750. &lt;p&gt;When you don&apos;t have a system proxy configured, the icon is greyed out.&lt;/p&gt;
  751. &lt;p&gt;&lt;picture&gt;
  752.  &lt;source srcset=&quot;https://blog.mastykarz.nl/assets/images/2024/02/proxystat-proxy-off.webp&quot; type=&quot;image/webp&quot;&gt;
  753.  &lt;img src=&quot;https://blog.mastykarz.nl/assets/images/2024/02/proxystat-proxy-off.png&quot; alt=&quot;ProxyStat showing a greyed out icon when system proxy is not configured&quot;&gt;
  754. &lt;/picture&gt;&lt;/p&gt;
  755. &lt;p&gt;When using tools like &lt;a href=&quot;https://aka.ms/devproxy&quot;&gt;Dev Proxy&lt;/a&gt;, Charles or Proxyman, it&apos;s easy to forget to turn off them off in your system proxy configuration. It can be frustrating to understand why your internet is not working and you can&apos;t open any website. ProxyStat is a great way to quickly see if you have a system proxy configured.&lt;/p&gt;
  756. &lt;p&gt;&lt;a href=&quot;https://mastykarzblog.blob.core.windows.net/downloads/ProxyStat.pkg&quot;&gt;Download ProxyStat&lt;/a&gt; for macOS 14.2 and configure it to automatically launch on startup.&lt;/p&gt;
  757. </description><pubDate>Sun, 04 Feb 2024 11:11:05 GMT</pubDate></item></channel></rss>

If you would like to create a banner that links to this page (i.e. this validation result), do the following:

  1. Download the "valid RSS" banner.

  2. Upload the image to your own server. (This step is important. Please do not link directly to the image on this server.)

  3. Add this HTML to your page (change the image src attribute if necessary):

If you would like to create a text link instead, here is the URL you can use:

http://www.feedvalidator.org/check.cgi?url=http%3A//feeds.feedburner.com/WaldekMastykarz

Home · About · News · Docs · Terms

 
Copyright © 2002-9 Sam Ruby, Mark Pilgrim, Joseph Walton, and Phil Ringnalda