236 replies to this topic

[DZ-Jay]

How big is your buffer, by the way? Are you keeping a command buffer along with the typeahead buffer? Or are you using the BACKTAB as your buffer? I always thought I'd use the BACKTAB and parse the string from it, rather than keeping track of it in memory.

[intvnut]

DZ-Jay, on Tue Feb 28, 2012 1:16 PM, said:

I get it. The queue is not really empty from the RUNQ perspective, but yield calls the idle task before queueing itself.

I wonder if it'll be better if P-Machinery were to just queue the input processor at the bottom of the state ISR, like it does with DOTIMER()...

Your thoughts?

If it's not too hard to just do it from an interrupt context, that would be best. While I think I've found the vast majority of "multi-frame" tokens and added appropriate "yield" calls, it's still safer to just run the task directly rather than queuing it. I don't want to get into a queue overflow situation.

[DZ-Jay]

intvnut, on Tue Feb 28, 2012 1:19 PM, said:

DZ-Jay, on Tue Feb 28, 2012 1:16 PM, said:

I get it. The queue is not really empty from the RUNQ perspective, but yield calls the idle task before queueing itself.

I wonder if it'll be better if P-Machinery were to just queue the input processor at the bottom of the state ISR, like it does with DOTIMER()...

Your thoughts?

If it's not too hard to just do it from an interrupt context, that would be best. While I think I've found the vast majority of "multi-frame" tokens and added appropriate "yield" calls, it's still safer to just run the task directly rather than queuing it. I don't want to get into a queue overflow situation.

Running from the ISR context is easy, but it increases the chance of re-entry. P-Machinery doesn't really like those...

[intvnut]

DZ-Jay, on Tue Feb 28, 2012 1:18 PM, said:

How big is your buffer, by the way? Are you keeping a command buffer along with the typeahead buffer? Or are you using the BACKTAB as your buffer? I always thought I'd use the BACKTAB and parse the string from it, rather than keeping track of it in memory.

I have buffers out the wazoo. I don't rely on the contents of BACKTAB for anything.

There's the typeahead buffer, a command input buffer, and a lexical analysis buffer, all separate. Overkill, certainly. But I still have almost 2K words of RAM left, and that's after blowing 1280 words on the scrollback.

Edited by intvnut, Tue Feb 28, 2012 1:27 PM.

[intvnut]

DZ-Jay, on Tue Feb 28, 2012 1:23 PM, said:

Running from the ISR context is easy, but it increases the chance of re-entry. P-Machinery doesn't really like those...

Gotcha. Maybe I just need a version of QTASK that refuses to queue a task that's already queued. Then, if there's a backlog, the queue won't fill up with redundant crap. Or a version of QTASK that drops a task if the queue's over a certain depth. Then you could safely queue the timer-driven tasks with that "QTASK.if_room" and never have to worry.

[intvnut]

intvnut, on Tue Feb 28, 2012 1:26 PM, said:

DZ-Jay, on Tue Feb 28, 2012 1:23 PM, said:

Running from the ISR context is easy, but it increases the chance of re-entry. P-Machinery doesn't really like those...

Gotcha. Maybe I just need a version of QTASK that refuses to queue a task that's already queued. Then, if there's a backlog, the queue won't fill up with redundant crap. Or a version of QTASK that drops a task if the queue's over a certain depth. Then you could safely queue the timer-driven tasks with that "QTASK.if_room" and never have to worry.

ie. something like this at the end of QTASK to make a version that only queues a task if the queue is half full or less:

@@if_room:  ; Only queue task if the queue is half full or less.
        MVI     TSKQHD, R2
        SUB     TSKQTL, R2
        ANDI    #TSKQM, R2      ; Slots used
        CMPI    #TSKQM/2+1, R2
        BLT     QTASK           ; There's room

        EIS
        JR      R5

Again, untested, but I always prefer to try to clarify things with code.

Edited by intvnut, Tue Feb 28, 2012 2:02 PM.

[revolutionika]

Hey, Intvnut, when you are done making Colossal Cave Adventure....what is your next game?

HEHEHEHEHE

[DZ-Jay]

intvnut, on Tue Feb 28, 2012 1:24 PM, said:

DZ-Jay, on Tue Feb 28, 2012 1:18 PM, said:

How big is your buffer, by the way? Are you keeping a command buffer along with the typeahead buffer? Or are you using the BACKTAB as your buffer? I always thought I'd use the BACKTAB and parse the string from it, rather than keeping track of it in memory.

I have buffers out the wazoo. I don't rely on the contents of BACKTAB for anything.

There's the typeahead buffer, a command input buffer, and a lexical analysis buffer, all separate. Overkill, certainly. But I still have almost 2K words of RAM left, and that's after blowing 1280 words on the scrollback.

Is that ECS or JLP Ram?

[DZ-Jay]

intvnut, on Tue Feb 28, 2012 2:02 PM, said:

intvnut, on Tue Feb 28, 2012 1:26 PM, said:

DZ-Jay, on Tue Feb 28, 2012 1:23 PM, said:

Running from the ISR context is easy, but it increases the chance of re-entry. P-Machinery doesn't really like those...

Gotcha. Maybe I just need a version of QTASK that refuses to queue a task that's already queued. Then, if there's a backlog, the queue won't fill up with redundant crap. Or a version of QTASK that drops a task if the queue's over a certain depth. Then you could safely queue the timer-driven tasks with that "QTASK.if_room" and never have to worry.

ie. something like this at the end of QTASK to make a version that only queues a task if the queue is half full or less:

@@if_room:  ; Only queue task if the queue is half full or less.
		MVI	 TSKQHD, R2
		SUB	 TSKQTL, R2
		ANDI	#TSKQM, R2	  ; Slots used
		CMPI	#TSKQM/2+1, R2
		BLT	 QTASK		   ; There's room

		EIS
		JR	  R5

Again, untested, but I always prefer to try to clarify things with code.

I wonder if that is necessary. It just shifts the constraint to another domain. If we have enough RAM, we may as well increase the size of the queue, no?

I have a patch for RUNQ and DOTIMER that I was planning on including in P-Machinery v2.0 that HLTs the CPU when an overflow occurs, but outputs the system state and the error when it happens. I use this while debugging Christmas Carol to try to catch bottlenecks that overflow the queue.

Likewise, calling the input scanner from the ISR context should not be a big problem. After all, the ISR won't be doing much game logic; it's all neatly packaged inside your interpreter engine.

What thinks ye?
-dZ.

[intvnut]

revolutionika, on Tue Feb 28, 2012 3:24 PM, said:

Hey, Intvnut, when you are done making Colossal Cave Adventure....what is your next game?

HEHEHEHEHE

Har de har har. :-) I do hope to hand this back off to Valter at some point. I want to integrate with what he's done so he can work on polishing it. You know, instead of starting work on 42 games in parallel....

DZ-Jay, on Tue Feb 28, 2012 4:21 PM, said:

intvnut, on Tue Feb 28, 2012 1:24 PM, said:

I have buffers out the wazoo. I don't rely on the contents of BACKTAB for anything.

There's the typeahead buffer, a command input buffer, and a lexical analysis buffer, all separate. Overkill, certainly. But I still have almost 2K words of RAM left, and that's after blowing 1280 words on the scrollback.

Is that ECS or JLP Ram?

JLP RAM. I have 8K words of JLP RAM. ECS only has 2K bytes.

Actually, I'm studiously avoiding using ECS RAM. I have good reason... but I hate to ruin the surprise. :-)

DZ-Jay, on Tue Feb 28, 2012 4:28 PM, said:

intvnut, on Tue Feb 28, 2012 2:02 PM, said:

intvnut, on Tue Feb 28, 2012 1:26 PM, said:

DZ-Jay, on Tue Feb 28, 2012 1:23 PM, said:

Running from the ISR context is easy, but it increases the chance of re-entry. P-Machinery doesn't really like those...

Gotcha. Maybe I just need a version of QTASK that refuses to queue a task that's already queued. Then, if there's a backlog, the queue won't fill up with redundant crap. Or a version of QTASK that drops a task if the queue's over a certain depth. Then you could safely queue the timer-driven tasks with that "QTASK.if_room" and never have to worry.

ie. something like this at the end of QTASK to make a version that only queues a task if the queue is half full or less:

@@if_room:  ; Only queue task if the queue is half full or less.
		MVI	 TSKQHD, R2
		SUB	 TSKQTL, R2
		ANDI	#TSKQM, R2	  ; Slots used
		CMPI	#TSKQM/2+1, R2
		BLT	 QTASK		   ; There's room

		EIS
		JR	  R5

Again, untested, but I always prefer to try to clarify things with code.

I wonder if that is necessary. It just shifts the constraint to another domain. If we have enough RAM, we may as well increase the size of the queue, no?

I have a patch for RUNQ and DOTIMER that I was planning on including in P-Machinery v2.0 that HLTs the CPU when an overflow occurs, but outputs the system state and the error when it happens. I use this while debugging Christmas Carol to try to catch bottlenecks that overflow the queue.

Likewise, calling the input scanner from the ISR context should not be a big problem. After all, the ISR won't be doing much game logic; it's all neatly packaged inside your interpreter engine.

What thinks ye?
-dZ.

Well, calling it from ISR context is perhaps ideal for Colossal Cave.

Going back to the fundamental queuing issues: For anything timer-based, you run the risk of a timer-based task overfilling the queue so that a self-retriggering task can't reschedule itself. While it's good to HLT on OVRFLO during development, for the final deployment it's probably worth considering a more graceful fallback. Recurring timer-based tasks are an easy target, because even if you drop one, it'll happen again soon.

If nothing else, anything timer based ought to leave enough room for anything self-retriggering, by design. I think the fact that one can break the other is a bug. Making the task queue larger just changes the threshold at which breakage happens. But, if we had a "queue fullness" threshold that was different for different classes of tasks, that'd allow us to leave room for the tasks that absolutely must be able to queue.

Thoughts?

[DZ-Jay]

intvnut, on Tue Feb 28, 2012 8:24 PM, said:

DZ-Jay, on Tue Feb 28, 2012 4:21 PM, said:

Is that ECS or JLP Ram?

JLP RAM. I have 8K words of JLP RAM. ECS only has 2K bytes.

Actually, I'm studiously avoiding using ECS RAM. I have good reason... but I hate to ruin the surprise. :-)

Oooh... I can't wait. I can't imagine what it is.

intvnut, on Tue Feb 28, 2012 8:24 PM, said:

Well, calling it from ISR context is perhaps ideal for Colossal Cave.

Going back to the fundamental queuing issues: For anything timer-based, you run the risk of a timer-based task overfilling the queue so that a self-retriggering task can't reschedule itself. While it's good to HLT on OVRFLO during development, for the final deployment it's probably worth considering a more graceful fallback. Recurring timer-based tasks are an easy target, because even if you drop one, it'll happen again soon.

If nothing else, anything timer based ought to leave enough room for anything self-retriggering, by design. I think the fact that one can break the other is a bug. Making the task queue larger just changes the threshold at which breakage happens. But, if we had a "queue fullness" threshold that was different for different classes of tasks, that'd allow us to leave room for the tasks that absolutely must be able to queue.

Thoughts?

Gotcha! I thought you were talking about any task, not timer-based ones. Having a threshold for different classes of tasks sounds like a good idea. However, I wouldn't want to extend the length of queue processing too much.

I'll also consider making the new version of P-Machinery scan on ISR or idle task, depending on a symbol defined by the programmer.


By the way, I was having problems with my browser yesterday while trying to reply so I don't know if I ever got to post this:

WOW! That demo you posted of the ESC behaviour is fantastic! Very nifty indeed!

-dZ.

Edited by DZ-Jay, Wed Feb 29, 2012 3:52 AM.

[intvnut]

DZ-Jay, on Wed Feb 29, 2012 3:47 AM, said:

intvnut, on Tue Feb 28, 2012 8:24 PM, said:

Well, calling it from ISR context is perhaps ideal for Colossal Cave.

Going back to the fundamental queuing issues: For anything timer-based, you run the risk of a timer-based task overfilling the queue so that a self-retriggering task can't reschedule itself. While it's good to HLT on OVRFLO during development, for the final deployment it's probably worth considering a more graceful fallback. Recurring timer-based tasks are an easy target, because even if you drop one, it'll happen again soon.

If nothing else, anything timer based ought to leave enough room for anything self-retriggering, by design. I think the fact that one can break the other is a bug. Making the task queue larger just changes the threshold at which breakage happens. But, if we had a "queue fullness" threshold that was different for different classes of tasks, that'd allow us to leave room for the tasks that absolutely must be able to queue.

Thoughts?

Gotcha! I thought you were talking about any task, not timer-based ones.

That wouldn't work too well... you wouldn't be able to fill the queue beyond halfway! :-)

DZ-Jay, on Wed Feb 29, 2012 3:47 AM, said:

Having a threshold for different classes of tasks sounds like a good idea. However, I wouldn't want to extend the length of queue processing too much.

Well, this wouldn't extend the time really. Rather, it just would drop tasks as we get overloaded. I'm not a huge fan of dropping tasks, but I can sleep OK at night dropping a 60Hz keyboard scanning task.

DZ-Jay, on Wed Feb 29, 2012 3:47 AM, said:

I'll also consider making the new version of P-Machinery scan on ISR or idle task, depending on a symbol defined by the programmer.

In fact, there's something like that already in RUNQ. I guess this would make the switch a little more automatic.

        ;; ---------------------------------------------------------------- ;;
        ;;  If the hand-controller scanner exists, call it to scan the      ;;
        ;;  controllers.  That is, unless we've been told not to.           ;;
        ;; ---------------------------------------------------------------- ;;
    IF  (DEFINED SCANHAND) = 1 AND (DEFINED RUNQ_NO_SCANHAND) = 0
        CALL    SCANHAND        ; Scan the hand controllers.
    ENDI

DZ-Jay, on Wed Feb 29, 2012 3:47 AM, said:

WOW! That demo you posted of the ESC behaviour is fantastic! Very nifty indeed!

Yeah, I thought it was pretty neat too. It fell pretty naturally out of the VFB implementation. I actually wasn't expecting it ahead of time. I noticed it mostly behaved that way during testing, since I set the top-of-scroll to 12 rows above the cursor whenever I wrap lines down. I added the same step to set top-of-scroll when I wrap lines backspacing and voila!

While Intellivision and "text adventure" may not seem like an immediate pairing for most people, I think this Adventure game is growing on me.

Edited by intvnut, Wed Feb 29, 2012 7:25 AM.